System and method for authorizing access to access-controlled environments

ABSTRACT

Systems and methods are provided for authorizing a user to access an access-controlled environment. The system includes a system server platform that communicates with fixed PC&#39;s, servers and mobile devices (e.g., smartphones) operated by users. The systems and methods described herein enable a series of operations whereby a user attempting to access an access-controlled environment is prompted to biometrically authenticate using the user&#39;s preregistered mobile device. Biometric authentication can include capturing images of the user&#39;s biometric features, encoding the features as a biometric identifier, comparing the biometric identifier to a previously generated biometric identifier and determining liveness. In addition, the authentication system can further authorize the user and electronically grant access to the access-controlled environment. In this manner the secure authentication system can, based on biometric authentication, authorize a user&#39;s access to devices, online services, physical locations or any networked environment that require user authorization.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is related to and includes U.S. PatentApplication Ser. No. 61/822,746, entitled “SYSTEM AND METHOD FORPROVIDING BIOMETRICALLY AUTHENTICATED ACCESS USING MOBILE DEVICES” filedMay 13, 2013; U.S. Patent Application Ser. No. 61/842,800, entitled“SYSTEM AND METHOD FOR PROVIDING BIOMETRICALLY AUTHENTICATED ACCESSUSING MOBILE DEVICES” filed Jul. 3, 2013; U.S. Patent Application Ser.No. 61/842,739, entitled “SECURE BACK-END ARCHITECTURE SYSTEM ANDMETHOD” filed Jul. 3, 2013; U.S. Patent Application Ser. No. 61/842,757,entitled “SYSTEM AND METHOD FOR GENERATING A BIOMETRIC IDENTIFIER” filedJul. 3, 2013; U.S. Patent Application Ser. No. 61/842,756, entitled“SYSTEMS AND METHODS FOR DETERMINING LIVENESS” filed Jul. 3, 2013; U.S.Provisional Patent Application Ser. No. 61/921,004, entitled “SYSTEM ANDMETHOD FOR DETERMINING LIVENESS” filed Dec. 26, 2013; U.S. ProvisionalPatent Application Ser. No. 61/920,985, entitled “SYSTEM AND METHOD FORGENERATING A BIOMETRIC IDENTIFIER” filed Dec. 26, 2013; U.S. ProvisionalPatent Application Ser. No. 61/922,438, entitled “SYSTEM AND METHOD FORBIOMETRIC PROTOCOL STANDARDS” filed Dec. 31, 2013; U.S. PatentApplication Ser. No. 61/924,092, entitled “SECURE BACK-END ARCHITECTURESYSTEM AND METHOD” filed Jan. 6, 2014; U.S. Patent Application Ser. No.61/924,097, entitled “SYSTEM AND METHOD FOR SMARTPHONE SECURITY CASE”filed Jan. 6, 2014; U.S. patent application Ser. No. 14/201,438,entitled “SYSTEMS AND METHODS FOR BIOMETRIC AUTHENTICATION

OF TRANSACTIONS” filed on Mar. 7, 2014; U.S. patent application Ser. No.14/201,462, entitled “SYSTEMS AND METHODS FOR DETERMINING LIVENESS”filed Mar. 7, 2014; and U.S. patent application Ser. No. 14/201,499,entitled “SYSTEM AND METHOD FOR GENERATING A BIOMETRIC IDENTIFIER” filedon Mar. 7, 2014, which are each hereby incorporated by reference as ifset forth in their respective entireties herein.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to systems and methods for providingauthenticated access, in particular, systems and methods for providingbiometrically authenticated access using a mobile device.

BACKGROUND OF THE INVENTION

There exist secure access systems for performing two-factorauthentication for a user to a network resource (e.g., remote servers,electronically locked access points, etc.). One exemplary two-factorauthentication system is The RSA SecurID® authentication mechanism byEMC Corporation of Bedford Mass. This exemplary system consists of a“token”—either hardware (e.g. a USB dongle) or software (a softtoken)—which is assigned to a computer user and which generates anauthentication code at fixed intervals (usually 60 seconds) using abuilt-in clock and the card's factory-encoded random key (known as the“seed record”). The seed record is different for each token, and isloaded into the corresponding system server as the tokens are purchased.The user authenticating must enter a personal identification number(PIN) and the generated authentication code being displayed at thatmoment.

However there have been the major breaches such systems, such as thebreach of RSA in 2011, culminating in the loss of sensitive data frommajor corporations to unknown sources. Knowing the seed records, affordsan attacker complete access to a user's information and access toanything they may use with their keys.

Pretty Good Privacy (PGP) is a data encryption and decryption computerprogram that provides cryptographic privacy and authentication for datacommunication. PGP can be used for signing, encrypting and decryptingtexts, e-mails, files, directories and whole disk partitions to increasethe security of e-mail communications. PGP, and other Private KeyEncryption methods, are very secure, as long as certain circumstancesremain true. In any private key exchange, if they private key is lost,stolen or misplaced, the user data is completely open. Conversely, ifthe user loses the key, the data they are protecting is lost forever.So, the tradeoff is apparent.

Numerous techniques have been proposed in the literature to deal withthe problem of identity theft. Any such scheme tries to establish that aperson is who she/he claims to be. Passwords, (long) private keys, andcamouflaging are some of the approaches used for this purpose. Sincehuman beings cannot remember long keys, private keys often tend to bestored in a wallet encrypted by possibly a small password.Unfortunately, all of these schemes have the property that someone whocarries these credentials (such as the right keys and passwords) will beaccepted as the right person even if these credentials have been stolenfrom others.

As a biometric is a biological characteristic (such as a fingerprint,the geometry of a hand, Retina pattern, iris shape, etc.) of anindividual, biometric techniques can be used as an additionalverification factor since biometrics are usually more difficult toobtain than other non-biometric credentials. Biometrics can be used foridentification and/or authentication (also referred to as identityassertion and/or verification).

Biometric identity assertion can require a certain level of security asdictated by the application. For example, authentication in connectionwith a financial transaction or gaining access to a secure locationrequires higher security levels. As a result, preferably, the accuracyof the biometric representation of a user is sufficient to ensure thatthe user is accurately authenticated and security is maintained.However, to the extent iris, face, finger, and voice identity assertionsystems exist and provide the requisite level of accuracy, such systemsrequire dedicated devices and applications and are not easilyimplemented on conventional smartphones, which have limited cameraresolution and light emitting capabilities.

The challenges surrounding traditional biometric feature capturetechniques, which generally require high resolution imagery,multi-spectral lighting and significant computing power to execute theexisting image analysis algorithms to achieve the requisite accuracydictated by security have made biometric authentication not widelyavailable or accessible to the masses. Moreover, traditional biometricauthentication techniques requiring dedicated devices used in a specificway (e.g., require a cooperative subject, have a narrow field of view,biometric must be obtained in a specific way) detracts from userconvenience and wide-scale implementation.

Accordingly, there is a need for systems and methods with which a user'sidentity can be verified conveniently, seamlessly, and with a sufficientdegree of accuracy, from biometric information captured from the userusing readily available smartphones. In addition, what is needed areidentity assertion systems and methods that, preferably, are not relianton multi-spectral imaging devices, multi-spectral light emitters, highresolution cameras, or multiple user inputs.

SUMMARY OF THE INVENTION

Technologies are presented herein in support of a system and method forauthorizing a user's access to an access-controlled environment.

According to a first aspect, a method for authorizing a user to accessan access-controlled environment includes the steps of receiving, by acomputing device having a storage medium having instructions storedtherein and a processor configured by executing the instructionstherein, access-control information that identifies theaccess-controlled environment. The method also includes accessing, bythe computing device, at least one database that includes user profilesthat include information to identify respective users, to identifyrespective mobile devices, and to identify respective transactionaccounts that are associated with respective access-controlledenvironments. In addition, the method includes receiving, by thecomputing device from a mobile device over a network, a transactionrequest including: a user identifier that identifies a user, and amobile device identifier that identifies the mobile device, wherein thetransaction request provides confirmation that the mobile device hasbiometrically authenticated the user. Furthermore, the method includesprocessing, by the computing device using the at least one database, thetransaction request to authorize the user to access theaccess-controlled environment by determining: that the user identifieris associated with at least one user profile stored in the at least onedatabase, that the mobile device identifier is associated with the atleast one user profile, and that the at least one user profileidentifies a transaction account associated with the access-controlledenvironment. The method also includes generating, by the computingdevice, an authorization notification that facilitates the authorizeduser to access to the access-controlled environment. In addition, themethod includes transmitting, by the computing device to at least oneremote computing device over a network, the authorization notification.

According to another aspect, a system is provided for authorizing accessto an access-controlled environment, the system including a networkcommunication interface, a computer-readable storage medium and one ormore processors configured to interact with the network communicationinterface and the computer-readable storage medium and execute one ormore software modules stored on the storage medium. The software modulesinclude a database module, that when executed configures the one or moreprocessors to access at least one database that includes user profilesthat include information to identify respective users, respective mobiledevices and respective transaction accounts that are associated withrespective access-controlled environments. The software modules alsoincludes a communication module that when executed configures the one ormore processors to receive access-control information that identifiesthe access-controlled environment, and to receive from a mobile deviceover a network, a transaction request including: a user identifier thatidentifies a user, and a mobile device identifier that identifies themobile device, wherein the transaction request provides confirmationthat the mobile device has biometrically authenticated the user. Thesoftware modules also includes an authorization module that that whenexecuted configures the one or more processors to process, using the atleast one database, the transaction request to authorize the user toaccess the access-controlled environment by determining: that the useridentifier is associated with at least one user profile stored in the atleast one database, that the mobile device identifier is associated withthe at least one user profile, and that the at least one user profileidentifies a transaction account associated with the access-controlledenvironment. The authorization module also configures the one or moreprocessors to generate an authorization notification that facilitatesthe authorized user to access to the access-controlled environment.Moreover, the communication module further configures the one or moreprocessors to transmit the authorization notification to at least oneremote computing device over a network.

These and other aspects, features, and advantages can be appreciatedfrom the accompanying description of certain embodiments of theinvention and the accompanying drawing figures and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level diagram of a system for authorizing access to anaccess-controlled environment in accordance with at least one embodimentdisclosed herein;

FIG. 2A is a block diagram of a computing device in accordance with atleast one embodiment disclosed herein;

FIG. 2B is a block diagram of computer software modules in accordancewith at least one embodiment disclosed herein;

FIG. 2C is a block diagram of a computing device in accordance with atleast one embodiment disclosed herein;

FIG. 3 is a flow diagram showing a routine for enrolling a useraccording to the user's biometric features in accordance with at leastone embodiment disclosed herein;

FIG. 4 is a flow diagram showing a routine for authorizing access to anaccess-controlled environment in accordance with at least one embodimentdisclosed herein;

FIG. 5 is a flow diagram showing a routine for authenticating a useraccording to the user's biometric features in accordance with at leastone embodiment disclosed herein;

FIG. 6A is a screenshot of an exemplary user interface in accordancewith at least one embodiment disclosed herein;

FIG. 6B is a screenshot of an exemplary user interface in accordancewith at least one embodiment disclosed herein; and

FIG. 7 is a flow diagram showing a routine for determining liveness inaccordance with at least one embodiment disclosed herein.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS OF THE INVENTION

By way of example only and for the purpose of overview and introduction,embodiments of the present invention are described below which concern asystem and method for authorizing a user's access to anaccess-controlled environment (ACE) according to the user's biometricfeatures.

In some implementations, the system includes a cloud based system serverplatform that communicates with fixed PC's, servers, and devices such aslaptops, tablets and smartphones operated by users. As the user attemptsto access a networked environment that is access-controlled, for examplea website which requires a secure login, the user is prompted toauthenticate using the user's preregistered mobile device.Authentication includes capturing biometric information in the form ofat least images of the user's eyes, periocular region and face or anycombination of the foregoing (collectively referred to as the Vitruvianregion), extracting unique features and encoding the features as aidentifier (“Vitruvian identifier”) using the mobile device. The systemcan also generate a unique mobile device identifier according to uniqueidentification information associated with the mobile device. The usercan then be authenticated according to the biometric information andmobile device information by either the mobile device or the systemserver or a combination of the two. User authentication can also includedetermining whether the biometric information and other non-biometricinformation indicates that user is a live subject, and not a picture orvideo reproduction attempting to spoof the system. If user issuccessfully authenticated, the system can electronically grant accessto the networked environment that the user's trying to access. Forexample, by transmitting an authorization notification to a third-partycomputing device. In this exemplary manner the secure authenticationsystem can be used to authenticate user access to websites, VPNs, accessat a physical door, access at an ATM, financial transactions or accessto any computer systems that require user identification/authentication.

The systems and methods of the present application provide significantconvenience for users as a function of biometrics-based accessmanagement and increased transaction security by supplementing oreliminating the need for passwords and/or devices dedicated to storinguser account information, such as cards or token key-fobs and the like.It is provided that many of the principles disclosed herein areapplicable to virtually any type of system requiring userauthentication, such as for example website access, physical accesscontrol, home, user role determination, group identification,automation, password management, or other access. The presentapplication removes a need for passwords, pins, tokens or the like fromany computing environment, including global computing networks.

According to a salient aspect of the subject application, capturingimages for the purpose of identifying a user's Vitruvian biometricfeatures can be performed using conventional digital cameras that arecommonly found on smart phones and other such mobile devices. Inaddition, identifying Vitruvian biometric features can be performedaccording to positive eye authentication techniques, preferably,applying algorithms analyzing the iris and/or periocular regions and/orface without requiring infra-red images or IR emitters which are notwidely integrated in smartphones.

According to a salient aspect of the subject application, biometricfeatures from the user's iris, periocular and/or facial regions can beextracted concurrently and seamlessly from common image captures (e.g.,the same image frames and same sequence of image frames captured),whereas, current identification techniques generally extract irisfeatures from certain image frames and periocular features from otherimage frames. Moreover, according to another salient aspect of thesubject application, Vitruvian biometric features are identified anddefined according to the spatial relationship of features (“keypoints”)within a single frame and the dynamic movement or position (“flow”) ofthose keypoints throughout a temporally arranged sequence of frames, soas to seamlessly generate an integrated Vitruvian biometric identifierof the user's Vitruvian region. The resulting integrated Vitruvianbiometric identifier is a single, virtual representation of the user'sVitruvian region, as opposed to, independently generating a plurality ofseparate biometric identifiers (e.g., one for the iris, another for theperiocular region) that are later fused. It can be appreciated that theidentifiers can be encoded as one or more vectors including thebiometric and non-biometric information.

The present disclosure also describes additional techniques forpreventing erroneous authentication caused by spoofing. In someexamples, the anti-spoofing techniques may include capturing multiplefacial images of a user, and analyzing the facial images for indicationsof liveness. A salient aspect of the subject application is that theprocess for generating a Vitruvian identifier that includes informationrelating to the dynamic movement of keypoints is representative ofliveness and/or can also be used to generate a liveness identifier.Using the liveness identifier, the disclosed system can determine“liveness” (e.g., whether the image sequence is of living user) anddetect suspected attempts to spoof by comparing the current livenessidentifier to a previously generated liveness identifier. In addition,liveness may be determined from analysis of the dynamic movement oflow-level Vitruvian features to determine if the flow is representativeof continuous motion. Liveness can also be indicated by the movement ofintermediate level features such as the eyes, mouth, and other portionsof the face. Such anti-spoofing programs may, in variousimplementations, detect facial movement based on specific areas of thehuman face. For example, the anti-spoofing programs may identify one orboth eyes of the facial image as landmarks. The anti-spoofing programsmay then detect and analyze transitions between the images as relates toone or both eyes. Using any detected transitions, the anti-spoofingprograms may detect facial gestures such as a blink, and the like. Basedon the analysis and the detection of a satisfactory, the livenessdetermination programs may prevent or grant access to functionalitiescontrolled by the computing device.

An exemplary system for authorizing access to an access-controlledenvironment 100 is shown as a block diagram in FIG. 1. In onearrangement, the system consists of a system server 105 and one or moreuser devices 101 including a mobile device 101 a and a computing device101 b. The system 100 can also include one or more remote computingdevices 102.

The system server 105 can be practically any computing device and/ordata processing apparatus capable of communicating with the user devicesand remote computing devices and receiving, transmitting and storingelectronic information and processing requests as further describedherein. Similarly, the remote computing device 102 can be practicallyany computing device and/or data processing apparatus capable ofcommunicating with the system server and/or the user devices andreceiving, transmitting and storing electronic information andprocessing requests as further described herein. It should also beunderstood that the system server and/or remote computing device can bea number of networked or cloud based computing devices.

In some implementations, computing device 102 can be associated with anenterprise organization that maintains user accounts and requireauthentication of account holders prior to granting access to securenetworked environments (e.g., secure website, bank, VPN, paymentproviders, and the like). The various types user's accounts used toaccess or interact with such networked environments are referred toherein as transaction accounts.

The user devices, mobile device 101 a and user computing device 101 b,can be configured to communicate with one another, the system server 105and/or remote computing device 102, transmitting electronic informationthereto and receiving electronic information therefrom as furtherdescribed herein. The user devices can also be configured to receiveuser inputs as well as capture and process biometric information, forexample, digital images and voice recordings of a user 124.

The mobile device 101 a can be any mobile computing devices and/or dataprocessing apparatus capable of embodying the systems and/or methodsdescribed herein, including but not limited to a personal computer,tablet computer, personal digital assistant, mobile electronic device,cellular telephone or smart phone device and the like. The computingdevice 101 b is intended to represent various forms of computing devicesthat a user can interact with, such as workstations, a personalcomputer, laptop computer, dedicated point-of-sale systems, ATMterminals, access control devices or other appropriate digitalcomputers.

As further described herein, the system for authorizing access to anaccess-controlled environment 100, facilitates the authentication of auser 124 according to a user's biometric features using a mobile device101 a. In some implementations, identification and/or authenticationaccording to a user's biometric features utilizes a user's biometricinformation in a two stage process. The first stage is referred to asenrollment. In the enrollment stage samples (e.g., images) ofappropriate biometric(s) is/are collected from an individual. Thesesamples of biometrics are analyzed and processed to extract features (orcharacteristics) present in each sample. The set of features present inthe biometric of an individual constitutes an identifier for the personand indicate whether the user is a live subject. These identifiers arethen stored to complete the enrolment stage. In the second stage thesame biometric of the individual is measured. Features from thisbiometric are extracted just like in the enrollment phase to obtain acurrent biometric identifier. If the goal is determining liveness, thefeatures or characteristics can be analyzed to determine if they arerepresentative of a live subject. If the goal is identification, thenthis identifier is searched for in the database of identifiers generatedin the first phase. If a match occurs, the identification of theindividual is revealed, otherwise identification fails. If the goal isauthentication, then the identifier generated in the second stage iscompared with the identifier generated in the first stage for theparticular person. If a match occurs, authentication is successful,otherwise authentication fails.

In some implementations, the system server can be configured to securelyfacilitate identification/authentication of the user's identity(collectively referred to as “identity assertion”) in furtherance of atransaction without authorizing the underlying transaction. In thismanner, the server is not required to retain the user's sensitivetransaction account information that is used to authorize the underlyingtransaction, instead, the system server is configured to authorize auser by recognizing one user from another at an appropriate level ofsecurity. For example, asserting the identity of a user conducting abank transaction according to the standards required by the bank andnotifying the bank's enterprise computing system (e.g., remote computingdevice 102) that the user has been authenticated. Accordingly, theexemplary systems and methods can supplement and/or replace the existingenterprise authentication processes by integrating with the existinginfrastructure and processes without interfering with establishedprocesses for authorizing the transactions once a user's identity hasbeen established for security purposes.

In addition to identity assertion, the system server 105 can alsoimplement additional security processes, including role gathering andaccess control, so as to facilitate authorization of requestedelectronic transactions or otherwise control a user's access. As such,the user authorization process can include identity assertion and canalso include authorization by determining whether the user's identity isassociated with one or more transaction accounts. In addition, thetransaction authorization process can also include determining theuser's level of access using the transaction account, for example,whether the user has the requisite permissions to perform requestedtransactions at an ATM.

In some implementations, the system server 105 can also implement rulesgoverning access to information and/or the transmission of informationbetween a variety of computing devices that users can interact with(e.g., mobile device 101 a, computing device 101 b) and one or moretrusted back-end servers (e.g., system server 105 and remote computingdevice 102). More specifically, the system server 105 can enforce rulesgoverning the user's access to information, as well as the sharing ofinformation with third-parties as authorized by the user. For example,the system server can regulate access to a database of informationpertaining to a user, and which has been biometrically authenticated bythe user, and limit access to that information according to rulesdefined by the user. By way of further example, maintaining a databaseof information and granting access to the information to anauthenticated user according to rules or permissions previously grantedto the user.

Exemplary systems and methods for facilitating identity assertion, rolegathering, access control and other security functions of the systemserver 105, including auditing and security assurance and accountabilityare further described herein and in co-pending and commonly assignedU.S. Provisional Patent Application Ser. No. 61/922,438, entitled“SYSTEM AND METHOD FOR BIOMETRIC PROTOCOL STANDARDS” filed Dec. 31,2013.

It should be noted that while FIG. 1 depicts the system for authorizingaccess to an access-controlled environment 100 with respect to a mobiledevice 101 a and a user computing device 101 b and a remote computingdevice 102, it should be understood that any number of such devices caninteract with the system in the manner described herein. It should alsobe noted that while FIG. 1 depicts a system 100 with respect to the user124, it should be understood that any number of users can interact withthe system in the manner described herein.

It should be further understood that while the various computing devicesand machines referenced herein, including but not limited to mobiledevice 101 a and system server 105 and remote computing device 102 arereferred to herein as individual/single devices and/or machines, incertain implementations the referenced devices and machines, and theirassociated and/or accompanying operations, features, and/orfunctionalities can be combined or arranged or otherwise employed acrossany number of such devices and/or machines, such as over a networkconnection or wired connection, as is known to those of skill in theart.

It should also be understood that the exemplary systems and methodsdescribed herein in the context of the mobile device 101 a are notspecifically limited to the mobile device and can be implemented usingother enabled computing devices (e.g., the user computing device 102 b).

In reference to FIG. 2A, the exemplary mobile device 101 a for use withthe system for authorizing access to an access-controlled environment100, includes various hardware and software components that serve toenable operation of the system, including one or more processors 110, amemory 120, a microphone 125, a display 140, a camera 145, an audiooutput 155, a storage 190 and a communication interface 150. Processor110 serves to execute a client application in the form of softwareinstructions that can be loaded into memory 120. Processor 110 can be anumber of processors, a central processing unit CPU, a graphicsprocessing unit GPU, a multi-processor core, or any other type ofprocessor, depending on the particular implementation.

Preferably, the memory 120 and/or the storage 190 are accessible by theprocessor 110, thereby enabling the processor to receive and executeinstructions encoded in the memory and/or on the storage so as to causethe mobile device and its various hardware components to carry outoperations for aspects of the systems and methods as will be describedin greater detail below. Memory can be, for example, a random accessmemory (RAM) or any other suitable volatile or non-volatile computerreadable storage medium. In addition, the memory can be fixed orremovable. The storage 190 can take various forms, depending on theparticular implementation. For example, the storage can contain one ormore components or devices such as a hard drive, a flash memory, arewritable optical disk, a rewritable magnetic tape, or some combinationof the above. Storage also can be fixed or removable.

One or more software modules 130 are encoded in the storage 190 and/orin the memory 120. The software modules 130 can comprise one or moresoftware programs or applications having computer program code or a setof instructions (referred to as the “mobile authentication clientapplication”) executed in the processor 110. As depicted in FIG. 2B,preferably, included among the software modules 130 is a user interfacemodule 170, a biometric capture module 172, an analysis module 174, anenrollment module 176, a database module 178, an authentication module180 and a communication module 182 that are executed by processor 110.

Such computer program code or instructions configure the processor 110to carry out operations of the systems and methods disclosed herein andcan be written in any combination of one or more programming languages.

The program code can execute entirely on mobile device 101, as astand-alone software package, partly on mobile device, partly on systemserver 105, or entirely on system server or another remotecomputer/device. In the latter scenario, the remote computer can beconnected to mobile device 101 through any type of network, including alocal area network (LAN) or a wide area network (WAN), mobilecommunications network, cellular network, or the connection can be madeto an external computer (for example, through the Internet using anInternet Service Provider).

It can also be said that the program code of software modules 130 andone or more computer readable storage devices (such as memory 120 and/orstorage 190) form a computer program product that can be manufacturedand/or distributed in accordance with the present invention, as is knownto those of ordinary skill in the art.

It should be understood that in some illustrative embodiments, one ormore of the software modules 130 can be downloaded over a network tostorage 190 from another device or system via communication interface150 for use within the system authorizing access to an access-controlledenvironment 100. In addition, it should be noted that other informationand/or data relevant to the operation of the present systems and methods(such as database 185) can also be stored on storage. Preferably, suchinformation is stored on an encrypted data-store that is specificallyallocated so as to securely store information collected or generated bythe processor executing the secure authentication application.Preferably, encryption measures are used to store the informationlocally on the mobile device storage and transmit information to thesystem server 105. For example, such data can be encrypted using a 1024bit polymorphic cipher, or, depending on the export controls, an AES 256bit encryption method. Furthermore, encryption can be performed usingremote key (seeds) or local keys (seeds). Alternative encryption methodscan be used as would be understood by those skilled in the art, forexample, SHA256.

In addition, data stored on the mobile device 101 a and/or system server105 can be encrypted using a user's biometric information, livenessinformation, or mobile device information as an encryption key. Forexample, using a key derivation function one or more secret keys can begenerated from unique user information such as biometric information.The key pair is therefore uniquely associated with the user by virtue ofbeing derived from the user's biometric information.

In some implementations, a combination of the foregoing can be used tocreate a complex unique key for the user that can be encrypted usingElliptic Curve Cryptography, preferably at least 384 bits in length, andstored on the mobile device. In addition, that key can be used to securethe user data stored on the mobile device and/or the system server.

Also preferably stored on storage 190 is database 185. As will bedescribed in greater detail below, the database contains and/ormaintains various data items and elements that are utilized throughoutthe various operations of the system and method for authenticating auser 100. The information stored in database can include but is notlimited to a user profile, as will be described in greater detailherein. It should be noted that although database is depicted as beingconfigured locally to mobile device 101 a, in certain implementationsthe database and/or various of the data elements stored therein can, inaddition or alternatively, be located remotely (such as on a remotedevice 102 or system server 105 not shown) and connected to mobiledevice through a network in a manner known to those of ordinary skill inthe art.

A user interface 115 is also operatively connected to the processor. Theinterface can be one or more input or output device(s) such asswitch(es), button(s), key(s), a touch-screen, microphone, etc. as wouldbe understood in the art of electronic computing devices. User Interfaceserves to facilitate the capture of commands from the user such as anon-off commands or user information and settings related to operation ofthe system for authenticating a user 100. For example, interface servesto facilitate the capture of certain information from the mobile device101 such as personal user information for enrolling with the system soas to create a user profile.

The computing device 101 a can also include a display 140 which is alsooperatively connected to processor the processor 110. The displayincludes a screen or any other such presentation device which enablesthe system to instruct or otherwise provide feedback to the userregarding the operation of the system for authenticating a user 100. Byway of example, the display can be a digital display such as a dotmatrix display or other 2-dimensional display.

By way of further example, the interface and the display can beintegrated into a touch screen display. Accordingly, the display is alsoused to show a graphical user interface, which can display various dataand provide “forms” that include fields that allow for the entry ofinformation by the user. Touching the touch screen at locationscorresponding to the display of a graphical user interface allows theperson to interact with the device to enter data, change settings,control functions, etc. So, when the touch screen is touched, userinterface communicates this change to processor, and settings can bechanged or user entered information can be captured and stored in thememory.

Mobile device 101 a also includes a camera 145 capable of capturingdigital images. The camera can be one or more imaging devices configuredto capture images of at least a portion of the user's body including theuser's eyes and/or face while utilizing the mobile device 101 a. Thecamera serves to facilitate the capture of images of the user for thepurpose of image analysis by the mobile device processor executing thesecure authentication application which includes identifying biometricfeatures for (biometrically) authenticating the user from the images.The mobile device 101 a and/or the camera 145 can also include one ormore light or signal emitters (not shown) for example, a visible lightemitter and/or infra-red light emitter and the like. The camera can beintegrated into the mobile device, such as a front-facing camera or rearfacing camera that incorporates a sensor, for example and withoutlimitation a CCD or CMOS sensor. Alternatively, the camera can beexternal to the mobile device 101 a. The possible variations of thecamera and light emitters would be understood by those skilled in theart. In addition, the mobile device can also include one or moremicrophones 104 for capturing audio recordings as would be understood bythose skilled in the art.

Audio output 155 is also operatively connected to the processor 110.Audio output can be any type of speaker system that is configured toplay electronic audio files as would be understood by those skilled inthe art. Audio output can be integrated into the mobile device 101 orexternal to the mobile device 101.

Various hardware devices/sensors 160 are also operatively connected tothe processor. The sensors 160 can include: an on-board clock to tracktime of day, etc.; a GPS enabled device to determine a location of themobile device; an accelerometer to track the orientation andacceleration of the mobile device; gravity magnetometer; proximitysensors; RF radiation sensors and other such devices as would beunderstood by those skilled in the art.

Communication interface 150 is also operatively connected to theprocessor 110 and can be any interface that enables communicationbetween the mobile device 101 a and external devices, machines and/orelements including system server 105. Preferably, communicationinterface includes, but is not limited to, a modem, a Network InterfaceCard (NIC), an integrated network interface, a radio frequencytransmitter/receiver (e.g., Bluetooth, cellular, NFC), a satellitecommunication transmitter/receiver, an infrared port, a USB connection,and/or any other such interfaces for connecting the mobile device toother computing devices and/or communication networks such as privatenetworks and the Internet. Such connections can include a wiredconnection or a wireless connection (e.g. using the 802.11 standard)though it should be understood that communication interface can bepractically any interface that enables communication to/from the mobiledevice.

At various points during the operation of the system authorizing accessto an access-controlled environment 100, the mobile device 101 a cancommunicate with one or more computing devices, such as system server105, user computing device 101 b and/or remote computing device 102.Such computing devices transmit and/or receive data to/from mobiledevice 101 a, thereby preferably initiating maintaining, and/orenhancing the operation of the system 100, as will be described ingreater detail below.

FIG. 2C is a block diagram illustrating an exemplary configuration ofsystem server 105. System server 105 can include a processor 210 whichis operatively connected to various hardware and software componentsthat serve to enable operation of the system for facilitating secureauthentication of transactions at a terminal 100. The processor 210serves to execute instructions to perform various operations relating touser authentication and transaction processing as will be described ingreater detail below. The processor 210 can be a number of processors, amulti-processor core, or some other type of processor, depending on theparticular implementation.

In certain implementations, a memory 220 and/or a storage medium 290 areaccessible by the processor 210, thereby enabling the processor 210 toreceive and execute instructions stored on the memory 220 and/or on thestorage 290. The memory 220 can be, for example, a random access memory(RAM) or any other suitable volatile or non-volatile computer readablestorage medium. In addition, the memory 220 can be fixed or removable.The storage 290 can take various forms, depending on the particularimplementation. For example, the storage 290 can contain one or morecomponents or devices such as a hard drive, a flash memory, a rewritableoptical disk, a rewritable magnetic tape, or some combination of theabove. The storage 290 also can be fixed or removable.

One or more software modules 130 (depicted in FIG. 2B) are encoded inthe storage 290 and/or in the memory 220. The software modules 130 cancomprise one or more software programs or applications (collectivelyreferred to as the “secure authentication server application”) havingcomputer program code or a set of instructions executed in the processor210. Such computer program code or instructions for carrying outoperations for aspects of the systems and methods disclosed herein canbe written in any combination of one or more programming languages, aswould be understood by those skilled in the art. The program code canexecute entirely on the system server 105 as a stand-alone softwarepackage, partly on the system server 105 and partly on a remotecomputing device, such as a remote computing device 102, mobile device101 a and/or user computing device 101 b, or entirely on such remotecomputing devices. As depicted in FIG. 2B, preferably, included amongthe software modules 130 are an analysis module 274, an enrollmentmodule 276, an authentication module 280, a database module 278, and acommunication module 282, that are executed by the system server'sprocessor 210.

Also preferably stored on the storage 290 is a database 280. As will bedescribed in greater detail below, the database 280 contains and/ormaintains various data items and elements that are utilized throughoutthe various operations of the system 100, including but not limited to,user profiles as will be described in greater detail herein. It shouldbe noted that although the database 280 is depicted as being configuredlocally to the computing device 205, in certain implementations thedatabase 280 and/or various of the data elements stored therein can bestored on a computer readable memory or storage medium that is locatedremotely and connected to the system server 105 through a network (notshown), in a manner known to those of ordinary skill in the art.

A communication interface 255 is also operatively connected to theprocessor 210. The communication interface 255 can be any interface thatenables communication between the system server 105 and externaldevices, machines and/or elements. In certain implementations, thecommunication interface 255 includes, but is not limited to, a modem, aNetwork Interface Card (NIC), an integrated network interface, a radiofrequency transmitter/receiver (e.g., Bluetooth, cellular, NFC), asatellite communication transmitter/receiver, an infrared port, a USBconnection, and/or any other such interfaces for connecting thecomputing device 205 to other computing devices and/or communicationnetworks, such as private networks and the Internet. Such connectionscan include a wired connection or a wireless connection (e.g., using the802.11 standard) though it should be understood that communicationinterface 255 can be practically any interface that enablescommunication to/from the processor 210.

The operation of the system for authorizing access to anaccess-controlled environment and the various elements and componentsdescribed above will be further appreciated with reference to the methodfor authenticating a user as described below, in conjunction with FIGS.3-4 with continued reference to FIGS. 1 and 2A-2C. The processesdepicted in FIGS. 3 and 4 are shown from the perspective of the mobiledevice 101 a as well as the system server 105, however, it should beunderstood that the processes can be performed, in whole or in part, bythe mobile device 101 a, the system server 105 and/or other computingdevices (e.g., remote computing device 102 and/or user computing device101 b) or any combination of the foregoing. It should be appreciatedthat more or fewer operations can be performed than shown in the figuresand described herein. These operations can also be performed in adifferent order than those described herein. It should also beunderstood that one or more of the steps can be performed by the mobiledevice 101 a and/or on other computing devices (e.g. computing device101 b, system server 105 and remote computing device 102).

FIG. 3 is a flow diagram illustrating a routine 400 for enrolling theuser 124 with the system 100. The enrollment process verifies the user'sidentity to ensure that the user is who they say they are and can alsospecify the manner in which the user 124 and the mobile device 101 a areidentified to the system server 105. In addition, enrollment can createa user profile which associates the user 124 with user devices (e.g.,user's mobile device 101 a and/or the user computing device 101 b) andwith one or more of the user's transaction accounts. Enrollment alsoincludes capturing (e.g., reading) the user's biometrics features,generating one or more biometric identifiers characterizing thosefeatures and determining the user's liveness. These steps can beperformed for verification as well as to establish a baseline for futureverification sessions as further described herein. Accordingly, it canbe appreciated that many of the steps discussed in relation to FIG. 3can be performed during subsequent user authentication sessions asdiscussed in relation to FIG. 4.

The process begins at step 305, where an initial communication sessionis established between the mobile device 101 a and the system server105. In some implementations, communications between the mobile deviceand system server can be established using 2-way Secure Socket Layers(SSL) established on a top of 1-way SSL communication. Morespecifically, the mobile device processor 110, which is configured byexecuting one or more software applications, including, preferably, thecommunication module 182 and the enrollment module 176, can transmit anAPI call to the system server 105 and establish a 1-way SSLcommunication session with the system server 105 in order to encrypt thecommunications. The API call can also include a private 2-way SSL key soas to establish a 2-way SSL secure communication environment. In someimplementations, the mobile device can transmit a pre-loaded 2-way SSLcertificate and an API key that is unique to the mobile device clientapplication. The pre-loaded certificate and key can be single useinstances that are stored when the client application is stored intomemory.

In addition, at step 305, the mobile device processor 110, which isconfigured by executing instructions in the form of one or more softwaremodules 130, including, preferably, the enrollment module 176, thecapture module 172, the communication module 182, the database module178, the analysis module 174, can also initialize the various componentsof the mobile device 101 a and determine their respective operabilityand capabilities.

Initialization can be performed during the initial enrollment processand can also be performed during subsequent biometriccapture/authentication processes. However, it should be understood thatsome or all of the steps need not be performed with each initializationand can be performed upon initial enrollment and/or periodicallythereafter. By way of non-limiting example, user enrollment andinitialization of a mobile device to facilitate biometric authenticationusing a mobile device are described herein and in co-pending andcommonly assigned U.S. Patent Application Ser. No. 61/842,800.

Then at step 310, the mobile device 101 a collects user identificationinformation. More specifically, the mobile device processor 110, whichis configured by executing one or more software modules 130, including,preferably, the enrollment module 176 and the user interface module 170,can prompt the user to input the user identification information andreceive the user inputs via the user interface 115. The useridentification information can include information about the user'sidentity (e.g., name, address, social security number, etc.). Forexample, as shown in FIG. 6A, the mobile device display 600 can promptthe user to input such personal information about the user's identity610. In some implementations, some or all of the information can begathered automatically from memory of the mobile device 101 a or from aremote computing device.

In addition, user identification information can include informationabout one or more transaction accounts with which the user desires toaccess one or more ACEs according to the systems and methods describedherein. For example, the user can enter pre-existing log-in andpasswords 615 associated with the user's various transaction accounts(e.g., online banking accounts, website log-ins, VPN accounts and thelike) or actual transaction account numbers 620 (e.g., bank accountnumbers, routing numbers, debit/credit card numbers, expiration datesand the like) as shown in FIG. 6A. In some implementations, theconfigured mobile device processor and/or the system server 105 canautomatically obtain some or all of such information directly from theenterprise organizations associated with the transaction accounts and/orACEs after verifying the user's identity according to the useridentification information provided by the user.

Then at step 315, mobile device identification information is collected.Mobile device identification information can include but is not limitedto at least a portion of the DeviceID, AndroidID, IMEI, CPU serialnumber, GPU serial number and other such identifiers that are unique tothe mobile device. More specifically, the mobile device processor 110,which is configured by executing one or more software modules 130,including, preferably, the enrollment module 176, can query the varioushardware and software components of the mobile device 101 a to obtainrespective device identification information. Using the mobile deviceidentification information the configured mobile device processor or thesystem server can generate one or more mobile device identifiers thatuniquely identify the mobile device as further described herein.

Then at step 320, the user's identity is verified. Identity verificationprovides additional security and determines that the user 124 is, infact, who they claim to be. It should be understood that verifying theuser's identity can be performed by the system server 105, the mobiledevice 101 a, or a combination of the foregoing.

For example and without limitation, the mobile device processor 110,which is configured by executing one or more software modules 130,including, preferably, the enrollment module 176 and communicationmodule 182, can transmit the user identification information can to thesystem server 105 for identity verification. In some implementations,the system server 105, can query a database storing the user's personaldata and determine whether the user information corresponds to thepreviously stored data. If the compared information does not correspondto a sufficient degree or additional user input is required, the systemserver can also generate follow up questions that are specific to theuser according to the database of personal data and forward thequestions to the mobile device 101 a thereby prompting the user 124 toinput answers to the questions using the mobile device. Various methodsof verifying a user's identity would be understood by those in the art.

In addition or alternatively, identity verification can also beperformed according to the mobile device information as furtherdescribed herein. For example, determining, by the system server 105,whether the user information and device information corresponds to themobile communication service account associated with the mobile device101 a as gathered from the mobile phone service provider's enterprisesystem.

In some implementations, the system server 105 can verify the user'sidentity according to transaction account and password information thatis already associated with one or more existing transaction accountsassociated with the user and stored on the system server or on a securedata store that is accessible by the system server. For example, if thesystem server is integrated with an existing enterprise security system,the user can be identified by, say, an existing account number and pinnumber or a log-in and password. In addition or alternatively the user'sidentity can be verified using third-party verification services, forexample, the Acxiom Personal Information Verification System by AcxiomCorp. of Little Rock, Ark.

It should be understood that the stringency of the identity verificationcan vary depending on the level of security as dictated by theparticular implementation of the secure authentication system 100. Forexample, user log-in to an online forum/discussion board might requireonly liberal verification of a user's identity, whereas applications inwhich the disclosed systems and methods are used to authenticate afinancial transaction may require stringent identity validation. Assuch, the identity verification can range from stringent verificationusing services such as Axciom to simply confirming whether the userlog-in and password match an existing log-in and password.

Then at step 325, if a user's identity is verified, a user profile canbe generated and stored. The user profile can include one or more piecesof user identification information and mobile device identification. Inaddition the user profile can include information concerning one or moreof the user's transaction accounts as well as settings that can be usedto guide the operation of the system 100 according to the user'spreferences.

In some implementations, the system server 105 can generate a uniqueidentifier for the user (a “userId”) and an associated mobile deviceidentifier (a “mobileld”) and store the identifiers in a clusteredpersistent environment so as to create the profile for the user. TheuserId and mobileld can be generated using one or more pieces of theuser identification information and mobile device identificationinformation, respectively. It should be understood that additional useridentification information and mobile device identification informationcan also be stored to create the user profile or stored in associationwith the user profile.

In addition, the userId and associated mobileld can be stored inassociation with information concerning one or more transaction accountsdescribed at step 315. In some implementations, the specific transactionaccount information can be stored on the system server 105 therebyenabling the system server to authorize all or part of the requestedtransactions on behalf of the user and the enterprise organization. Inaddition or alternatively, the user profile can be associated with atransaction account using, for example, an identifier (e.g., a site IDor global unique identifier, etc.) or other such pointer to a securedatastore storing the sensitive transaction account information, say,the remote computing device 102 operated by an enterprise organization.Accordingly, the system server 105 is not required to store sensitivetransaction account information, and, as further described herein, thesystem server 105 can generate and/or forward requests to authorize auser to the appropriate enterprise organization for further processing.In addition or alternatively, the system server can query the securedatastore to gather the necessary information for processing any suchrequests.

At this juncture, it can be appreciated that the userId can be used tomap the user profile to the user's legacy transaction accounts. Inaddition, the mobileld ties the device to a user profile. In someimplementations, the userIds are a convention, whereas, the mobileldsare mandatory because the mobileld alone can link the user 124 andmobile device 101 a pair to the user profile maintained by the systemserver 105 and/or the user's transaction accounts. Moreover, anyadditional information included in the user profile can be used fornon-repudiation or provenance purposes by the system server 105 infuture authorization requests.

It can be appreciated that user profiles can be created by the systemserver 105 and/or the mobile device 101 a. Moreover, one or moreinstances of a user profile can be stored on various devices (e.g.,system server 105, mobile device 101 a, remote computing device 102, oruser computing device 101 b). In addition, the information included inthe various instances of the user's profiles can vary from device todevice. For example, an instance of the user profile which stored on themobile device 101 a can include the userId, mobileld, useridentification information and sensitive information concerning theuser's transaction accounts, say, account numbers and the like. By wayof further example, the instance of the user profile stored by thesystem server 105 can include the userId, mobileld, other uniqueidentifiers assigned to the user and information that identifies theuser's transaction accounts but does not include sensitive accountinformation.

In some implementations, generating the user profile by the systemserver 105 can also include generating a private key, say, a unique2-way SSL certificate using the user identification information, whichcan include the information concerning the user's transactionaccount(s), and the mobile device identification information. Thegenerated private key can also be transmitted back to the mobile device101 a for storage in the mobile device. Accordingly, the generated keycan be used for subsequent communications in conjunction with identityassertion sessions.

For example, the enrollment/genesis phase can link the information thatidentifies the user (e.g., userId, SSN, email, or other useridentifiers) to a Common Name (CN) which can be the particular manner inwhich the particular user is uniquely identified by the system server105 and/or legacy transaction account systems in the two way securesocket layers key. Accordingly. the genesis phase can also link legacytransaction accounts associated with the user (e.g., the user's bankaccount) with the user identity maintained by the system server 105.

The private key is generated on the system server 105 and links themobile device 101 a (e.g., mobileId) and user (e.g., userId) pair to theuser identity (e.g., user identifier, common name, etc.) that will beused for subsequent communication.

The identity as asserted through the 2-way Secure Socket Layer key canbe maintained for all communication. This key is encoded with a passwordknown to only the device being used during enrollment, which in thisexample is the mobile device 101 a. In addition, the key isprogrammatically placed in the key store on the mobile device 101 a. Itis the only mechanism allowing the identity and links to the genesisphases. No human or device knows the password used to encrypt the 2-waySSL key. Accordingly, the mobile device 101 a, using the private key,has an identity to offer in subsequent communications. It can beappreciated that each enabled mobile device associated with a user canhave a unique key that can be linked to the same user profile enablingthe use of multiple devices in the same manner. In addition oralternatively, separate user profiles can be established and maintainedfor each user-device pair independently or in a linked fashion. It canalso be appreciated that, similarly, multiple users can use the samedevice(s) which correspond to individual user profiles or joint userprofiles or otherwise linked user profiles.

Accordingly, as a result of genesis/enrollment, a user profile iscreated which associates the user 124, the mobile device 101 a and oneor more transaction accounts. In addition, the mobile device 101 a canbe provided with information (e.g., a unique user identifier and mobiledevice identifier and/or unique keys) for identifying the user 124 andmobile device 101 a in subsequent communications, say, identityassertion sessions.

Then at step 330, user settings are received. Settings includepreferences and rules defined by the user for guiding the operation ofthe system 100. In some implementations, during the enrollment processor at any point thereafter, the mobile device 101 a can prompt the userto input settings and associate those settings with one or more of theuser's transaction accounts. The settings can be stored by the mobiledevice or the system server 105 or a combination of the foregoing.Accordingly, the user defined settings can cause the system 100 toauthenticate the user and/or facilitate transactions automatically, orwith fewer user inputs.

In some implementations, the user input settings can specify preferredaccess-controlled environments that the user desires to access using thesystem. For example, settings can identify certain websites orapplications that the user wishes to automatically log-into using thesystem 100. In some implementations the settings can specifycircumstances in which a user desires to authenticate for gaining accessto such environments. For example, the user desires to authenticate onlywhen making a purchase through a particular mobile-application, asopposed to authenticating immediately upon launching the particularmobile application.

In some implementations, the user settings can specify preferences forconducting transactions. For example and without limitation, the usercan specify default payment methods/accounts thereby configuring themobile device 101 a and/or the system server 105 to select transactionaccounts and/or process transactions efficiently. In addition, the usercan associate the payment methods with specified merchants. By way offurther example, a user can specify rules to control use of certaintransaction accounts, say, causing the system server 105 to preventcertain types of transactions, cause a notification to be provided tothe user or implement additional security measures to ensure approvedaccount usage.

In some implementations, the user settings can include user definedaccess rules or privacy settings controlling access to the user'sinformation or activity or accounts. For example, the settings canidentify other enrolled users or enterprise organizations that the userdesires to have access to the user's accounts or information associatedwith the user.

In some implementations, the settings can specify default transactionrules for conducting transactions with defined enterprise organizations.For example, settings can specify that the user typically wishes towithdraw a prescribed amount of cash from a default transaction accountwhen conducting an ATM transaction. Accordingly the system 100 canautomatically conduct the transaction by applying the user definedsettings when a transaction at an ATM is initiated without requiring theuser to provide or confirm the transaction account and transactiondetails.

In some implementations, a user can also set one-time transaction rulesin advance of conducting certain electronic transactions. For example,the user can specify that the next time the user accesses a financialinstitution's network, the user desires to make a $500 payment into theuser's account held with the enterprise organization using a particularpayment method. In this manner, the user can queue a number of differenttransactions to be carried out automatically by the system 100.

It should be understood that the described settings are presented asnon-limiting examples, and that a wide variety of settings can be usedto control the operation of the system 100 and how users interact withthe system 100.

It should be also understood that during enrollment and any timethereafter and while using any user devices (e.g., mobile device 101 aand user computing device 101 b) that are enrolled with the system, theuser can adjust settings regarding user's preferences for interactingwith the system 100. For example, the mobile device can receive from theuser additional user identification information, passwords, transactionaccount information and the like for storage locally on the mobiledevice 101 a, on the system server 105, the user computing device 101 bor a combination of the foregoing. As such, any of the computing devicesof the system 100 can be configured to act as a platform forautomatically facilitating access to ACEs using such transactionaccounts and providing the user's information to the various enabledcomputing devices (e.g., mobile device 101 a, user computing device 101b, remote computing device 102).

Then at step 335, the user's biometrics features are captured using themobile device 101 a. In some implementations, the mobile deviceprocessor 110, which is configured by executing one or more softwaremodules 130, including, preferably, the enrollment module 176, theanalysis module 174, the user interface module 170, and the capturemodule 172, prompts the user to capture imagery of the user'siris/irises, eye(s), periocular region, face (e.g., the Vitruvianregion) or a combination of the foregoing using the mobile device camera145 and stores a sequence of images to storage 190 or memory 120.

In some implementations, the configured processor 110 can also cause themicrophone 104 to capture the user's voice through a microphone incommunication with the mobile device and record the audio data to thedevice memory. For example, the user can be prompted to say words orphrases which are recorded using the microphone. The mobile device canalso capture images of the user's face, eyes, etc. while recording theuser's voice, or separately.

Then at step 340, one or more biometric identifiers are generated fromthe captured biometric information and are stored to complete theenrolment stage. More specifically, the mobile device processor 110,which is configured by executing one or more software modules 130,including, preferably, the capture module 172, the database module 178,the analysis module 174, can analyze the biometric information capturedby the camera and generate a biometric identifier (e.g., “a Vitruvianidentifier”) as further described herein and in reference to FIG. 5.

In some implementations, the user's voice biometric features can becharacterized as a voice print such that the user can be biometricallyauthenticated from characteristics of the user's voice according tovoice speaker identification algorithms. For example, the audiocomponent of the user's biometric information can be analyzed by themobile device processor according to the voice speaker identificationalgorithms to create a voice print for the user which can be stored bythe mobile device. The various technologies used to process voice data,generate and store voice prints can include without limitation,frequency estimation, hidden Markov models, Gaussian mixture models,pattern matching algorithms, neural networks, matrix representation,vector quantization and decision trees. Accordingly, the user can beauthenticated/identified or liveness determined by analyzing thecharacteristics of the user's voice according to known voice speakeridentification algorithms as further described herein.

In some implementations, the configured mobile device processor 110 candetermine if the biometric information captured is sufficient togenerate adequate biometric identifiers. If the biometric features arenot identified with sufficient detail from the biometric informationcaptured (e.g., imagery, audio data, etc.), the configured mobile deviceprocessor can prompt the user to repeat the biometric capture processvia the display or other such output of the mobile device 101 a. Inaddition, the configured mobile device processor 110 can providefeedback during and after capture thereby suggesting an “idealscenario”, for example and without limitation, a location with adequatevisible light, the appropriate distance and orientation of the camerarelative to the user's face and the like.

Moreover, in some implementations, the configured mobile deviceprocessor can analyze the light captured by the camera and the lightspectrum that can be emitted by light emitters on the mobile device, andadjust the frequency of the light emitted during the capture step so asto improve the quality of the biometric information captured by thecamera. For example, if the configured processor is unable to generate abiometric identifier, and determines that the user has darker coloredeyes, the processor can cause the camera to recapture the image data andcause the light emitter to emit light frequencies that are, say, asclose to the infra-red spectrum as possible given the particular mobiledevice's capabilities so as to capture more features of the user's iris.

In addition to generating the one or more biometric identifiers asdiscussed above, the configured mobile device processor can alsogenerate identifiers incorporating multiple instances of one or morebiometric identifiers. For example, during the enrollment process, theconfigured mobile device processor can capture and analyze multiplesequences of biometric information so as to generate multiple biometricidentifiers that, collectively, are adequate virtual representations ofuser 124 across the multiple captures (e.g., to ensure that theconfigured processor has “learned” enough biometric information for user124). Accordingly, the biometric capture portion of the enrollmentprocess can be performed several times at various intervals andlocations so as to capture the user's biometric information in variousreal-world scenarios, thereby increasing the likelihood that futureauthentication will be positive and without error. It should beunderstood that the multiple biometric identifiers can be storedseparately and/or combined into a single identifier.

In addition or alternatively, complex biometric identifiers can begenerated by fusing identifiers generated according to differentbiometric identification modalities to create a multi-dimensionalbiometric identifier that is a combined biometric representation of theuser. For example, the mobile device processor configured by executingone or more modules including, preferably, the analysis module 174, cancombine the user's voice print(s) and the Vitruvian identifier(s).

In some implementations, the biometric identifiers can be stored locallyon the mobile device 101 a in association with the user's profile suchthat the mobile device can perform biometric authentication according tothe biometric identifiers. In addition or alternatively, the biometricidentifiers can be stored in association with the user's profile on aremote computing device (e.g., system server 105 or remote computingdevice 102) enabling those devices to perform biometric authenticationof the user.

At step 345, the mobile device processor 110, which is configured byexecuting one or more software modules 130, including, preferably, thecapture module 172, can also receive non-machine-vision basedinformation. Non-machine-vision based information generally relates tobehavioral characteristics of the user 124 during enrollment andsubsequent authentication sessions that are indicative of the user'sidentity as well as the user's liveness. For example and withoutlimitation, non-machine-vision based information can include a timereceived from an on-board clock, a location received from GPS device,how far from the user's face the camera is positioned during imagecapture calculated from imagery or other on-board proximity measuringdevices, the orientation of the mobile device and acceleration of themobile device received from an accelerometer, RF radiation detected byan RF detector, gravity magnetometers which detect the Earth's magneticfield to determine the 3-dimensional orientation in which the phone isbeing held, light sensors which measure light intensity levels and thelike.

In some implementations, the non-machine-vision based information isreceived over time and stored such that the configured processor candetermine patterns in the information that are unique to the user 124 byapplying behavioral algorithms as would be understood by those in theart. Accordingly, during later authentication stages, the currentnon-computer-vision based data collected can be analyzed and compared tothe user's established behavioral traits to verify the user's identityas well as determine whether the information is indicative of liveness.For example, time and location based behavioral patterns can beidentified over time and the current position compared to the pattern todetermine if any abnormal behavior is exhibited. By way of furtherexample, the particular “swing” or acceleration of the mobile deviceduring multiple authentication processes can be characterized as abehavioral trait and the particular swing of the current authenticationcan be compared to identify abnormal behavior. By way of furtherexample, the device orientation or distance from the user's face canalso be similarly compared. By way of further example, an RF radiationsignature for the user can be established during enrollment and comparedto future measurements to identify abnormal RF radiation levels (e.g.,suggesting the use of video screens to spoof the system).

At step 350, the mobile device processor configured by executing one ormore software modules 130, including, preferably, the analysis module174, can generate one or more liveness identifiers which characterizethe captured user's biometrics and/or the non-machine-vision basedinformation that are indicative of the user's liveness. As noted above,determining liveness is an anti-spoofing measure that can be performedduring enrollment and subsequent authentication sessions to ensure thatthe image sequence captured by the imaging device is of a live subjectand not a visual representation of the user by, say, a high resolutionvideo. In some implementations liveness is determined by detectingmovement of biometric features because every time the user enrolls orvalidates the user will actually move a little no matter how steadyhe/she is trying to be.

In some implementations, the process for generating biometricidentifiers, as discussed at step 335 and process 500 of FIG. 5, canused to generate a liveness identifier and/or determine the user'sliveness. More specifically, the configured mobile device processor,employing the steps of process 500, can extract and record dynamicinformation of Vitruvian biometric features and encode the features as abiometric identifier that is indicative of liveness and/or as a uniqueliveness identifier. In addition, it should be understood that theconfigured processor can analyze the dynamic information to identifyfluid motion of the features within the image sequence that areindicative liveness. More particularly, liveness can be determined fromanalysis of the dynamic movement of low-level Vitruvian features todetermine if the flow is representative of continuous motion. Similarly,liveness can also be determined by the movement of intermediate levelfeatures such as the eyes, mouth, and other portions of the face.

In addition or alternatively, the configured processor can generate aliveness identifier and/or determine liveness according to the Eulerianmotion magnification algorithms which is also referred to as Eulerianvideo magnification (EMM or EVM). EMM can be used to amplify smallmotions of the subject captured in the images, for example, flushing ofthe subject's skin during a heartbeat. In some implementations, whenemploying EMM, the camera (e.g., the smartphone camera) and the subjectare still, however, the configured processor can use EMM to detect thesesmall motions of the subject even while the device is moving using videostabilization.

In some implementations, a liveness identifier can be generated and/orliveness determined, by analyzing lip movement, pupil dilation,blinking, and head movement throughout the image sequence. Moreover, aliveness identifier can also be generated and liveness determined byanalyzing the audio recording of the user voice, as would be understoodby those skilled in the art. Moreover, in some implementations, livenesscan also be determined from analyzing the light values associated withlow level, intermediate and/or high level features represented in asingle image and/or throughout multiple image frames in the sequence todetermine abnormal light intensities in the frame(s).

In addition, the non-machine-vision based information including, timereceived from an on-board clock, location received from a gps device,how far from the user's face the camera is positioned during imagecapture as calculated from imagery received from the camera or otheron-board distance measuring device, the mobile device orientation duringfeature acquisition, acceleration of the mobile device while the mobiledevice is drawn into position for acquisition as received from anaccelerometer can all be used to generate an identifier characterizingthe user's unique behavioral characteristics and/or analyzed todetermine if the information is indicative of liveness during enrollmentand authentication sessions.

It should be understood that one or more liveness identifiers generatedaccording to the image based and non-machine-vision based methods can beanalyzed and stored individually or combined to generate one or moremulti-dimensional biometric and/or liveness identifiers.

Then at step 355, the one or more biometric identifiers and one or moreliveness identifiers are stored. In some implementations, the mobiledevice processor, which is configured by executing one or more softwaremodules 130, including, preferably, the enrollment module 176 and thedatabase module 178, can store the biometric identifiers and livenessidentifiers locally, so as to perform biometric authentication on themobile device 101 a, thereby avoiding transmission of the sensitivebiometric information to the system server for storage.

In some implementations, the configured mobile device processor cantransmit the biometric identifiers, liveness identifiers and otherinformation (e.g., a generated mobileId) to the system server 105 as oneor more data packets, for example, as described in co-pending andcommonly assigned U.S. Patent Application Ser. No. 61/842,800, entitled“SYSTEM AND METHOD FOR PROVIDING BIOMETRICALLY AUTHENTICATED ACCESSUSING MOBILE DEVICES” filed Jul. 3, 2013. It should be understood thatadditional user and mobile device specific information (e.g., useridentification information), can also be transmitted to the systemserver so as to associate the one or more liveness identifiers,biometric identifiers and mobile device identifiers with a particularuser.

It should be understood that some or all of the enrollment process stepscan be repeated using other user devices, e.g., user computing device 10lb. For example, a unique mobileld can be generated for other userdevices used in conjunction with the system 100, thereby enabling userauthorization using multiple enrolled user devices.

Turning now to FIG. 4, which is a flow diagram that illustrates aroutine 400 for authorizing a user to access an ACE in accordance withat least one embodiment disclosed herein.

The process begins at step 405 where the mobile device 101 a is promptedto authenticate the user 124. In some implementations, the mobile deviceis prompted to authenticate by receiving a user input. For example, theuser can launch the secure authentication client application whichdisplays a prompt 630 on the touchscreen 600 of the mobile devicerequesting the user to input whether they would like to authenticateusing virtual buttons 635, as shown in FIG. 6B. In some implementations,the mobile device 101 a can begin the authentication processautomatically. For example, the mobile device can prompt the user toauthenticate upon detecting that the user has used the mobile device toaccess an ACE requiring user authorization as specified by the usersettings or by the enterprise organization that operates the ACE.

In some implementations, the system server 105 can cause the mobiledevice 101 a to begin authentication in response to receiving anauthorization request. Preferably, the authorization request includesaccess-control information that identifies the ACE. In addition, theauthorization request, preferably, identifies the user 124 and or anassociated user computing device thereby enabling the system server 105to cause the appropriate user's mobile device to commenceauthentication. More specifically, in response to the authorizationrequest, the system server 105 can cross-reference the user and/orcomputing device identified in the request with database of userprofiles to determine whether the user or device is associated with auser profile and, hence, is enrolled with the system. Likewise, thesystem server can determine whether the user profile identifies anenrolled mobile device and transmit a biometric authentication requestto the identified mobile device thereby prompting the mobile device tobiometrically authenticate the user.

By way of example and without limitation, the authorization request canbe received by the system server directly from a remote computing device102 that controls access to the ACE (e.g., a financial institutioncomputing system, a networked computing device that controls anelectronic door lock providing access to a restricted location, aweb-server that requires user authentication prior to allowing the userto access a website). By way of further example, the request forauthentication can be received by the system server 105 from a usercomputing device (e.g., computing device 101 b) that is being used togain access to a networked environment. In this example, the usercomputing device 101 b can act as the intermediary to the ACE back-endserver by transmitting the authorization request to the system server105, receiving responses from the system server and relaying informationto the ACE server so as to facilitate access to the ACE. In addition oralternatively, the system server can communicate directly with the ACEback-end servers in accordance with the disclosed embodiments.

Then at step 410, the mobile device processor 110, which is configuredby executing one or more software modules, including, the authenticationmodule 180, the user interface module 170, the analysis module 174 andthe capture module 172, captures the user's current biometricinformation. In addition, the configured processor can also capturecurrent non-machine-vision based information as well as current mobiledevice identification information. The capture of such information canbe performed by the mobile device in the manner described in relation tosteps 315, 335 and 345 of FIG. 3 and as further described herein inrelation to FIG. 5.

Then at step 415, the mobile device processor 110, which is configuredby executing one or more software modules, including, the authenticationmodule 180 and the analysis module 174, generates one or more currentbiometric identifiers in the manner described in relation to steps 340of FIG. 3 and as further described herein in relation to FIG. 5.

Then at step 420, the mobile device processor 110, which is configuredby executing one or more software modules, including, the authenticationmodule 180, the user interface module 170, the analysis module 174, cangenerate one or more current liveness identifiers using the currentbiometric information and/or current non-machine-vision basedinformation in the manner described in relation to steps 335-350 of FIG.3 and as further described herein in relation to FIG. 5.

In addition, at step 425, the mobile device processor 110, which isconfigured by executing one or more software modules, including, theauthentication module 180, the user interface module 170, the capturemodule 172 and the analysis module 174, can extract the mobile deviceidentification information that is currently associated with the mobiledevice 101 a and generate a current mobile identifier substantially inthe same manner as described in relation to steps 315 and 325 of FIG. 3.It should be understood that such information and a mobile deviceidentifier need not be generated with each authentication session. Insome implementations, a previously generated identifier, say, themobileld generated during initial enrollment, can be used to identifythe mobile device.

Then at step 430, the user is authenticated according to at least aportion of the one or more current biometric identifiers. Using thecurrent biometric identifiers, the user's identity can be authenticatedby comparing the biometric identifiers to one or more stored biometricidentifiers that were previously generated during the enrollment processor subsequent authentication sessions. It should be understood that thebiometric authentication step is not limited to using the exemplaryVitruvian biometric identifiers and can utilize any number of otherbiometric identifiers generated according to various biometricidentification modalities (e.g., iris, face, voice, fingerprint, and thelike).

In some implementations, the mobile device processor, configured byexecuting one or more software modules 130, including, preferably, theauthentication module, authenticates the user 124 by matching at least aportion of the one or more current biometric identifiers generated atstep 515 to the previously generated version(s) and determining whetherthey match to a requisite degree. For example, the configured mobiledevice processor can apply a matching algorithm to compare at least aportion of the current biometric identifiers to the stored versions anddetermine if they match to a prescribed degree. More specifically, in anexemplary matching algorithm, the process of finding frame-to-frame(e.g., current identifier to stored identifier) correspondences can beformulated as the search of the nearest neighbor from one set ofdescriptors for every element of another set. Such algorithms caninclude but not limited to the brute-force matcher and Flann-basedmatcher.

The brute-force matcher looks for each descriptor in the first set andthe closest descriptor in the second set by comparing each descriptor(e.g., exhaustive search). The Flann-based matcher uses the fastapproximate nearest neighbor search algorithm to find correspondences.The result of descriptor matching is a list of correspondences betweentwo sets of descriptors. The first set of descriptors is generallyreferred to as the train set because it corresponds to a pattern data(e.g., the stored one or more biometric identifiers). The second set iscalled the query set as it belongs to the “image” where we will belooking for the pattern (e.g., the current biometric identifiers). Themore correct matches found (e.g., the more patterns to “image”correspondences exist) the more chances are that the pattern is presenton the “image.” To increase the matching speed, the configured processorcan train a matcher either before or by calling the match function. Thetraining stage can be used to optimize the performance of theFlann-based matcher. For this, the configured processor can build indextrees for train descriptors. And this will increase the matching speedfor large data sets. For brute-force matcher, generally, it can storethe train descriptors in the internal fields.

In addition, at step 435, the user is further authenticated by verifyingthe user's liveness. In some implementations, liveness of the user canbe determined by comparing at least a portion of the one or more currentliveness identifiers generated at step 420 with the previously generatedversions and determining whether they match to a requisite degree. Asnoted above, verifying the user's liveness can also include analyzingthe captured biometric and non-machine-vision information and/or theliveness identifier(s) to determine whether they exhibit characteristicsof a live subject to a prescribed certainty. In some implementations,the configured processor 110 can analyze the dynamic information encodedin the liveness identifier to determine if the information exhibitsfluid motion of the biometric features within the image sequence thatare indicative of a living subject. More particularly, liveness can bedetermined from analysis of the dynamic movement of low-level Vitruvianfeatures to determine if the flow is representative of continuousmotion. Similarly, liveness can also be determined by the movement ofintermediate level features such as the eyes, mouth, and other portionsof the face. Similarly, liveness can be determined by comparing themovement of the user's intermediate level features with one or moreother biometric characterizations of the user to determine if theycorrespond. For example, the user's lip movements can be compared to theuser's voice print to determine whether the lip movement corresponds tothe words spoken by the user during the capture process at step 410.

Whether liveness is determined by matching liveness identifiersaccording to a matching algorithm or by analyzing the informationcaptured at step 410 or liveness identifiers generated at step 420 forindicators of liveness can be dependent on environmental constraints,for example, lighting. More specifically, if the biometric informationis captured in poor lighting conditions, liveness can be determinedusing matching algorithms. Alternatively, if the biometric informationis captured under adequate lighting conditions, liveness can bedetermined by analyzing the captured information and/or the generatedidentifiers which characterize the biometric information.

Moreover, the current non-computer-vision based information collected atstep 410 can also be analyzed and compared to the user's establishedbehavioral traits to determine whether they match to a prescribeddegree. For example, time and location based behavioral patterns can beidentified over time and the current position compared to the pattern todetermine if any differences (e.g., abnormal behavior) are exhibited. Byway of further example, the particular “swing” or acceleration of themobile device during multiple authentication processes can becharacterized as a behavioral trait and the particular swing of thedevice during the current authentication session can be compared toidentify abnormal behavior. Similarly, the device orientation ordistance from the user's face can also be compared. It should beunderstood that this analysis can be performed to determine liveness aswell as to authenticate the user's identity in connection with step 435.Exemplary systems and methods for determining liveness are furtherdescribed herein and in co-pending and commonly assigned U.S. patentapplication Ser. No. 14/201,462, entitled “SYSTEMS AND METHODS FORDETERMINING LIVENESS” filed Mar. 7, 2014.

Then, at step 440, the user is authorized by the system server 105.Authorization can include verifying that an enrolled user who has beenbiometrically authenticated using an enrolled mobile device isattempting to access the ACE.

In some implementations, the mobile device processor 110, which isconfigured by executing one or more software modules 130, includingpreferably, the authentication module 180 and the communication module182, can generate at least one transaction request and transmit thetransaction request to the system server 105. For example and withoutlimitation, the transaction request can include: information identifyingthe user (e.g., user identification information or a user identifiergenerated during authentication or enrollment); information identifyingthe mobile device (e.g., mobile device identification or a mobile deviceidentifier generated during authentication or enrollment); informationindicating whether the user has been biometrically authenticated; andinformation concerning the ACE that the user is attempting to access.

In some implementations, the transaction request can include a private2-way SSL key generated during the enrollment process and thatestablishes a 2-way SSL secured communication session between the mobiledevice 101 a and the system server 105. The key can include informationidentifying the user and mobile device, for example, a user identifierand a mobile device identifier. In addition or alternatively, the keycan include information that is useable to identify the user-mobiledevice pair. It should be understood that the transaction request and/orthe information included in the transaction request(s) can betransmitted as a number of separate transmissions. Similarly, theprocessing of the request as further described at step 445 can beperformed in any number of steps by the mobile device 101 a, or thesystem server 105, or the remote computing device 102, or a combinationof the foregoing.

In response to receipt of the transaction request, the system server105, using a processor 210 which is configured by executing one or moresoftware modules 130, can process the transaction request to authorizethe user to access the ACE. For example, the system server cancross-reference the user identified in the transaction request with adatabase of user profiles to determine whether the user is associatedwith a user profile and, hence, is enrolled with the system 100.Likewise, the system server can determine whether the mobile deviceidentified by the request is also associated with the user profile. Insome implementations, the user can be authorized by comparing thereceived key to one or more keys stored in association with respectiveuser profiles to identify a match, thereby verifying that the userand/or mobile device identified by the key corresponds to a user profilestored in the database.

In addition, the step of authorizing the user can also includedetermining, by the system server, whether the transaction requestindicates that the user has been biometrically authenticated. In someimplementations, verifying biometric authentication can includedetermining whether the transaction request conforms to a predeterminedconfiguration. For example, the transaction request can be generated bythe mobile device only upon successful biometric authentication of theuser by the mobile device. Accordingly, receipt of the transactionrequest provides confirmation that the user has been biometricallyauthenticated. By way of further example, the transaction request can begenerated to include the key that is useable to identify the user and/ormobile device only upon successful biometric authentication. By way offurther example, the transaction request can include additionalindicators, flags, session information and the like which indicate thatthe user has been biometrically authenticated and can also provideadditional security to the authenticity of the transmission.

Similarly, it should be understood that as all transmissions to and fromthe various computing devices (e.g., mobile device 101 a, user computingdevice 101 b system server 105, and remote computing device 102) can betime stamped and time sensitive and/or include communication sessioninformation. As such, the authorization process can also be contingentupon authentication occurring within a pre-defined duration or “time tolive” from the time-stamp of each packet of data being sent to thesystem server. In the event of a malformed or MITM (man in the middle)type assault, where a packet was redesigned, the time-to-live providesadditional security since it would be challenging to rebuild a newpacket with correct data within the amount of time the TTL is set to.

Authorization can also include determining, by the system server 105,whether the user has permission to access the ACE and/or conduct atransaction (e.g., access a secure website or perform a financialtransaction, or access stored information etc.). Preferably, during theauthorization process, the system server 105 receives access-controlinformation identifying the ACE. For example, in the scenario where themobile device automatically initiates authentication upon detecting thatthe user is attempting to access an ACE, the transaction request caninclude the access-control information identifying the ACE. By way offurther example, if an authorization request is received from the systemserver from a remote computing device associated with an ACE, theauthorization request can include the access-control information. Basedon the ACE identified in the access-control information, the systemserver 105 can determine whether the user's profile identifies one ormore transaction account(s) that are useable to access the ACE.

In some implementations, the transaction request, the authorizationrequest and/or the access-control information received by the systemserver 105 can include transaction details describing the nature of therequested user access and/or a particular transaction to be conductedbetween the user and the ACE. Accordingly, user authorization by thesystem server 105 can include further authorizing access and/orauthorizing the particular transaction. More specifically, the systemserver 105 can query one or more defined data stores to gather anyaccess rules (e.g., access permissions, roles, settings etc.) associatedwith one or more of the user's transaction accounts and which governaccess using the one or more transaction accounts. Likewise, the systemserver can also gather access rules which govern access to the ACE.Based on the so gathered access rules and transaction details, thesystem server can determine whether the user is authorized to access theACE and/or perform the requested transaction.

Then at step 445, an authorization notification is generated accordingto whether the user is authorized to access the ACE at step 440. In someimplementation, the system server 105 can transmit the authorizationnotification directly to the ACE that the user is attempting to accessor indirectly via one or more computing devices being used by the userto access the ACE (e.g., mobile device 101 a or user computing device101 b). For example, the authorization notification can be transmittedto a remote computing device 102 that controls access to the ACE andtherefore requires the user authorization (e.g., a networked computingdevice that controls an electronic door lock providing access to arestricted location, a server that requires user authorization prior toallowing the user to access a private website or a secure data-store, aATM terminal requiring authorization prior to dispensing funds). By wayof further example, the authorization notification can be transmitted tothe mobile device 101 a or the user computing device 101 b with whichthe user is attempting to gain access to an ACE using a transactionaccount. Based on the authorization notification, any such remotecomputing device which receives the authorization notification can grantaccess to the user and/or further authorize the user to access the ACEand/or process the requested transaction.

The substance and form of the authorization notification can varydepending on the particular implementation of the system 100. Forexample, in the case of user attempting to access a website, thenotification can simply identify the user and indicate that the userbeen biometrically authenticated and the user's identity has beenauthorized/verified. In addition or alternatively, the notification caninclude information concerning one or more transaction accounts, say,the user's log-in and password information or a one-time password. Inother instances, say, when user is trying to complete a financialtransaction, the notification can include the user's payment data,transaction details and the like. In some implementations, theauthorization notification can include a fused key, which is a one-timeauthorization password that is fused with one or more biometric, user,mobile device, or liveness identifiers, user identification informationand/or mobile device identification information, and the like. In suchan implementation, the computing device receiving the authorizationnotification can un-fuse the one time password according tocorresponding identifiers previously stored by the remote computingdevice and utilize the encoded information to grant access to the user.

Turning now to FIG. 5, a flow diagram illustrates a routine 500 fordetecting the user's biometric features from a series of images inaccordance with at least one embodiment disclosed herein and generatinga biometric identifier for the purposes of authenticating a user and/ordetermining the user's liveness. In general, the routine includescapturing and analyzing one or more images, preferably an imagesequence, of at least the user's eyes, periocular region and surroundingfacial region (collectively referred to as the facial region or theVitruvian region); identifying low-level spatiotemporal features from atleast the eyes and periocular regions for the purposes of generating anidentifier that compresses the low-level spatiotemporal features (theVitruvian biometric identifier). As compared to high level features,which generally characterize the overall image frame (e.g., the entirepicture of the user's facial region), or intermediate features, whichcharacterize objects within the greater image frames (e.g., the nose),low-level features are frequently used to represent imagecharacteristics and in this case biometric characteristics. Low-levelfeatures are preferable in that they are robust for imagecharacterization in that they provide invariance under rotation, size,illuminosity, scale and the like.

The inclusion of the periocular region in generating a biometricidentifier can be beneficial in that in images where the iris featuresalone cannot be reliably obtained (or used), the surrounding skin regionmay be used to characterize the user's biometric features which can beused to effectively confirm or refute an identity. Moreover, the use ofthe periocular region represents a balance between using the entire faceregion and using only the iris for recognition. When the entire face isimaged from a distance, the iris information is typically of lowresolution and the extraction of biometric features from the irismodality alone will be poor.

Furthermore, the collective aggregation of low-level periocular featureseffectively generates a Vitruvian identifier characterizing higher levelfeatures, e.g., intermediate level features. The periocular region canbe considered to be an intermediate level feature with high performancewhen it comes to classification of the subject, because, in general, theperiocular region provides a high concentration of unique features fromwhich a user can be classified (biometrically).

It should be understood that, according to the disclosed embodiments,the images can be captured and the biometric identifier that isindicative of the user's identity and/or liveness can be generated usingmobile devices (e.g. smartphones) that are widely available and havingdigital cameras capable of capturing images of the Vitruvian region inthe visible spectral bands. However, it should be understood that thedisclosed systems and methods can be implemented using computing devicesequipped with multispectral image acquisition devices that can image inboth the visible and near-IR spectral bands. Such multispectral imageacquisition user devices can facilitate capturing the iris texture andthe periocular texture.

The process begins at step 505, where the mobile device processor 110configured by executing one or more software modules 130, including,preferably, the capture module 172, causes the camera 145 to capture animage sequence of at least a portion of the user's (124) Vitruvianregion and stores the image sequence in memory. Capturing the imagesequence includes detecting, by the mobile device camera 145, lightreflected off a portion of the user's Vitruvian region. Preferably, theportion of the user's Vitruvian region includes the user's iris/irises,eye(s), periocular region, face or a combination of the foregoing. Inaddition, the configured processor can cause the mobile device to emitlight, at least in the visible spectrum, to improve the intensity of thereflection captured by the camera. In addition, although not required,the mobile device can also be configured to emit infra-red light toaugment the spectrum of reflected light that is captured by the camera.It should be understood that the image sequence includes a plurality ofimage frames that are captured in sequence over a period of time.

Then at step 510, a first image frame is analyzed and low-level featuresare identified and their relative positions recorded. More specifically,the mobile device processor 110 configured by executing the softwaremodules 130, including, preferably, the analysis module 172, analyzes afirst individual image frame to extract/detect spatial information ofthe low-level Vitruvian biometric features including, preferably,periocular features. The configured processor can detect the features or“keypoints” by executing a keypoint detection algorithm including butnot limited to, SIFT, SURF, FREAK, Binary features, Dense SIFT, ORB orother such algorithms whether known in the art or new. The configuredprocessor encodes each of the keypoints detected using the pixel values(e.g., how bright and what color the pixel is) that correspond to theidentified keypoint thereby defining a local key descriptor. Theselow-level features generally range from 3 to approximately 100 pixels insize, however it should be understood that low-level features are notlimited to falling within the aforementioned range. Similar to mostimage algorithm's descriptors (SIFT, SURF, FREAK, etc.), the set ofpixels does not necessarily represent a square area. Each feature'scomputation entails thorough histogram estimations that are taken, forexample, over 16×16 regions. It should be understood that the size ofthe histogram or region can be considered to represent the strength ofthe feature and is a non-linear function of pixels (e.g. it is notnecessarily a function of image quality).

Then at step 515, a continuous series of subsequent frames are analyzedand spatial and/or dynamic information of the keypoints identified atstep 510 is extracted. Using the keypoint descriptors encoded/generatedat step 510, the mobile device processor 110, which is configured byexecuting the software modules 130, including, preferably, the analysismodule 172, analyzes a plurality of subsequent frames to identify thecorresponding keypoints in each of the subsequent images in the sequenceof images. More specifically, the pixels defining the local keypointdescriptors are detected in the subsequent image frames and spatial anddynamic information for the detected pixels is extracted. Such dynamicinformation includes the relative movement of the pixels throughout theseries of pixel image frames. For example, the configured processor cananalyze the next, say, 5-10 frames in the image sequence by applying analgorithm (e.g. Lukas Kanade or Brox algorithms and the like) to detectthe pixels corresponding to the keypoints in each of the images in thesequence. The configured processor can track the position of a sparse ordense sample set of pixels throughout the frames and record thepositions.

The relative position (e.g. movement) of a pixel from one image frame toanother is referred to as the “optical flow displacement” or “flow”. Itshould be understood that the optical flow displacement can also besampled using other multi-frame, recursive analysis methods.

The configured processor can quantize the total amount of points bypopulating them spatially and temporally in histogram bins that can beencoded in the memory of the mobile device. Wherein each bin representshow much ‘optical flow’ and spatial ‘gradients’ exist in the clusters ofpixels associated with a particular keypoint descriptor.

Preferably, the configured processor can populate the histograms,according to algorithms, including but not limited to, HOOF, HOG or SIFTand the like. Accordingly, the paths can be defined as histograms oforiented gradients (temporal or spatial) and histograms of orientedflows.

Temporal gradients represent the change in position over time(direction, magnitude, time between the image frames) e.g., flow of apixel or pixels. For example, a pixel intensity identified in the firstimage frame that is then identified at another pixel location in asecond image frame in the sequence, can be expressed as a temporalgradient. Spatial gradients represent the difference of intensitiesaround a particular pixel or groups of pixels in an image frame. Forexample, the intensity of a pixel X in a first image frame and theintensity of surrounding pixels X−1, X+1, Y−1, Y+1, can be representedas a oriented gradient showing the difference in intensity between X andsurrounding pixels X−1, X+1, etc. By way of further example, a blackpixel right next to a white pixel that is right next to a black pixel isa very strong gradient whereas three white pixels in a row have nogradient.

Accordingly, both spatial and temporal information is defined in thehistograms. Coupling such spatial information and temporal informationenables a single Vitruvian characterization to be both a function ofsingle image content as well as of dynamic motion content over timethroughout multiple images.

It should be understood that one or more pre-processing operations canbe performed on the image frames prior to performing steps 510 and 515.By example and without limitations, pre-processing on the image dataprior to analysis can include scaling, orienting the image frames incoordinate space and the like as would be understood by those skilled inthe art.

It should also be understood that additional pre-processing operationscan be performed by the configured processor on the spatial and temporalinformation before populating the information in the histograms. Byexample and without limitation, pre-processing can include, Computingalgebraic combinations of the derivatives of the tracked flow paths,deepr, spatial derivative textures, motion boundary histograms akin toInria CVPR 2011, Kalman, filters, stabilization algorithms and the like.

Then at step 520, the salient pixel continuities are identified. Themobile device processor 110, which is configured by executing thesoftware modules 130, including, preferably, the analysis module 172,can identify salient pixel continuities by analyzing the “optical flow”of the pixels throughout the sequence of frames and recorded in thehistograms.

In general, the path of movement of one or more pixels can be analyzedand compared to prescribed criteria in order to determine whatcharacteristic the flow exhibits (e.g., is flow representative of astatic pixel, a continuously changing position, of non-fluid motion suchas jumping around the image frame, etc.). Preferably, the salient pixelcontinuities are those pixels and groups of pixels that have opticalflow values that are continuous.

More specifically, the configured processor can compare the optical flowgradients of a pixel to a prescribed set of continuity criteria whichare defined to ensure the presence of flow dynamics. For example andwithout limitation, continuity criteria can include but is not limitedto, the presence of deeper derivatives on the flow tracks of the pixeldefining a particular keypoint. By way of further example, continuitycriteria can be established through analysis of image sequences capturedof live subjects to identify optical flow values/characteristicsexhibited by live subjects as compared to flow values/characteristicsexhibited by imagery taken of non-live subjects. It should be understoodthat these characteristics can be unique to the user or can becharacteristics shared by other live subjects. If the pixel associatedwith a particular keypoint has flow that meets the continuity criteriathe particular pixel can be identified as salient continuities. In otherwords, if the pixel exhibits flow that meets the continuity criteria,the pixel or group of pixels can be determined to indicate liveness. Ifpixels showing liveness are found, then the processor can determine thatthe subject of the images is alive, hence, determining liveness, asfurther described herein.

It should be understood that, because histogram bins are essentiallydistributions of pixel areas, the configured processor can analyze flowon a pixel by pixel basis or greater groups of associated pixels (e.g.,multiple pixels defining a particular keypoint).

Then at step 525, Vitruvian primitives can be computed based on, amongother things, the salient pixel continuities identified at step 520. TheVitruvian primitives are computational constructs that characterize aparticular user's Vitruvian region according to the spatial arrangementof features identified at step 510 and dynamic information identified at515. More specifically, the primitives are computed, using theconfigured mobile device processor, on the space of the histogramdistributions. Because the space of histograms can be verycomputationally expensive and mobile devices are generally not ascomputationally powerful as traditional biometric authenticationsystems, the Vitruvian primitives can be computed on the space ofhistograms thereby resulting in histograms that are lower incomputational complexity.

In some implementations, the configured processor can expand the spatialkeypoint binning to higher algebraic combinations of gradient forms,thereby resulting on all possible spatiotemporal distributions of binnedquantities. The configured processor can compute the features in a shortspatiotemporal domain, for example, up to 5 pixel image frames. However,it should be understood that shorter or longer spatiotemporal domain canbe used. For example, when applying Eulerian coupling a longer domain ispreferable.

Then at step 530, the Vitruvian primitives are stored by the configuredprocessor in the memory of the mobile device as a Vitruvian identifier.In addition, the configured processor can generate and store one or morebiometric identifiers which includes at least the Vitruvian identifier.

It should be understood that while routine 500 is described in referenceto generating a Vitruvian identifier, such terms should not beinterpreted as limiting, as the routine 500 is applicable to theextraction and characterization of any number of biometric features fromimagery of any portion(s) of an individual's body, including but notlimited to, the user's face, eyes (including the iris) and/or periocularregion to define a biometric identifier. Moreover, the routine 500 isalso applicable to the identification and characterization of featuresfrom imagery of non-human subjects.

It can also be appreciated that, in addition to characterizing a user bygenerating a Vitruvian identifier according to routine 500 as describedabove, additional biometric features can be extracted from the imagesequence captured at step 505, or captured separately from step 505.Such additional biometric features can include by way of example andwithout limitation, soft biometric traits. “Soft biometric” traits arephysical, behavioral or adhered human characteristics as opposed to hardbiometrics such as fingerprints, iris, periocular characteristics andthe like which are generally invariant. However, it should be understoodthat certain features within the periocular region can offer informationabout features that can be used as soft biometrics, such as eye-shape.By way of further example, soft biometric traits can include physicaltraits such as skin textures, or skin colors. Soft biometrics can alsoinclude motion as detected by smartphone gyroscope/accelerometer, eyemotion characteristics as detected by eye tracking algorithms and headmotion characteristics as detected by tracking the movement of a faceand/or head.

Such biometric features can be extracted and characterized according tothe foregoing method as well as existing biometric analysis algorithms.In addition, the additional characterizations of the user's biometricfeatures can be encoded as part of the Vitruvian identifier concurrentlyto execution of the exemplary routine 500, or otherwise included in abiometric identifier which includes the Vitruvian identifier, forexample by fusing the soft biometric identifiers with the Vitruvianidentifier.

It should also be understood that the biometric identifier is notlimited to including the exemplary Vitruvian identifier and can includeany number of alternative biometric representations of a user such asidentifiers generated according to known biometric identificationmodalities (e.g., iris, face, voice, fingerprint, and the like).

According to another salient aspect of the subject application, thebiometric identifier that is generated by, among other things,extracting dynamic information selecting salient pixel continuities andrecording the temporal gradients e.g., ‘flow’ characterizes the user'sbiometric features and is also indicative of the liveness of the user.Accordingly, in addition to generating a Vitruvian identifier that isalso indicative of liveness, process 500 can also be implemented todetermine liveness and/or generate a liveness identifier for thepurposes of determining the liveness of user. As such, the configuredmobile device processor employing one or more of the steps of process500, can extract and record dynamic information of local key points inthe images, and analyze the dynamic information to, at a minimum,identify salient continuities that exhibit flow to define a livenessidentifier. It should be understood that the liveness identifier can beseparate from or integrally incorporated into the Vitruvian identifiergenerated by exemplary process 500. As such, references to livenessidentifier can be interpreted as a distinct identifier or the Vitruvianidentifier.

In addition, as discussed above in relation to FIGS. 3-5 and furtherdiscussed herein, liveness can be determined by differentiating betweena real face and an attempt to spoof the authentication process using,for example, a photograph or video of a face.

Some liveness detection systems attempt to distinguish between realfaces and ‘spoof photographs and videos by analyzing the image qualityof the face. For example, photographs and videos may have a lowercontrast ratio than that of a real face, or may have a lower resolutionand therefore appear less sharp. However, it can be difficult for acamera to identify such differences if the spoof print is also of highimage quality. Other liveness detection systems check that the face islive by requesting the user to make actions on request, for example byasking the user to blink at a certain time. A disadvantage of thistechnique is the user's actions must be interrupted to pass the test. Assuch, liveness detection systems that can reliably operate withoutrequiring actions by the user can be beneficial.

In accordance with the disclosed embodiments, liveness can be determinedbased on one or more reflectivity characteristics of the imagerycaptured by the mobile device camera, for example, by illuminating theface by using light from the display or a light emitter, and determiningthat the reflectivity characteristics of one or more images captured bythe camera are consistent with that of a real face, and/or that thereflectivity characteristics of the camera image are not consistent withthat of a photograph or video display or other object.

Turning now to FIG. 7, a flow diagram illustrates a routine 700 fordetecting the user's liveness from one or more images in accordance withat least one embodiment disclosed herein using, for example, a mobiledevice 101 a having a processor 110 which is operatively connected toone or more light emitters. In some implementations, the light emitterscan be light emitting diodes (LEDs) that can emit light in the visiblespectrum, infra-red (IR) spectrum, near-IR (NIR) spectrum and the likeor any combination of the foregoing. Systems and methods for determiningliveness based on reflectivity characteristics of images of facialfeatures (e.g., from the eyes, skin, cornea, and the like) are furtherdescribed herein and in co-pending and commonly assigned U.S. patentapplication Ser. No. 14/201,462, entitled “SYSTEMS AND METHODS FORDETERMINING LIVENESS” filed Mar. 7, 2014.

In order to more reliably distinguish a user's real eye from animpostor, say, a high resolution print of the user's eye (e.g.,‘spoofing’) the mobile device processor can capture imagery of theuser's eyes/face and analyze the images to ensure reflectioncharacteristics particular to a human cornea are present in the capturedimage. In some implementations, this can be done by pulsing theintensity of one or more of the LEDs, and capturing imagery whilepulsing the LEDs using the camera (step 710). In the case of a printedcornea reflection the reflection will be continuously present in theimages captured, in the case of the genuine cornea, the reflectionsdepicted in the images will pulsate as the LED does. Accordingly, byanalyzing the reflections, the mobile device processor can distinguishbetween reflections of the LED from a genuine cornea and a print thatincludes an image of a reflection in the cornea.

In a preferred embodiment, one of the LEDs remains continuously on andone of the NIR LEDs is pulsated at 3 Hz with its intensity varyingsinusoidally; and the camera has a frame rate of more than 12 frames persecond (fps). Preferably, the camera captures multiple image frames foranalysis, for example, 30 images. The processor can then analyze thecaptured images and select, the one or more images having the highestimage quality (e.g. bright and unblurred) to be used for iris patternrecognition so as to identify the user (step 715). All of the images, ora subset, can be used to detect the presence of cornea reflections anddetermine liveness as further described herein.

In order to detect reflections, the processor can align the images sothat all images of the iris occur at the same position in each image(step 720). It can be appreciated that the aligned images provide datarelating to the intensity of the iris spatially (like a photograph), andtemporally (like a video).

Then, at step 725, for each pixel spatially, the processor can processthe temporal intensity data to determine the magnitude of the frequencycomponent at 3 Hz, and divide this by the magnitude of the frequencycomponent at 0 Hz. For example, this can be performed by the processorusing a Goertzel filter. As a result, the processor can generate animage that shows the strength of the reflection from the pulsating LEDcompared to the strength of the reflection from the continuous LED (step730). As can be understood by those in the art, the physical compositionof a genuine eye/cornea does not reflect the same amount of light as anon-genuine reproduction nor do they reflect light in exactly the samemanner. Accordingly, the processor can then analyze the resulting imageto determine if the reflection intensities are indicative of a genuinecornea or a reproduced cornea (step 735). In the case of a printed eyebeing imaged, the resulting image can have generally constant intensityand of about 50% intensity of a genuine cornea. In the case of a genuinecornea (e.g., captured from a live subject) the resulting image shouldexhibit a sharp peak of high intensity corresponding to the reflectionthat is only created by the pulsating LED and not the continuous LED. Inaddition, the processor can also detect differences in intensity due toshadows created in the periocular region, which give an additionalindication that the acquired image has a 3D profile and hence is a livesubject.

In addition, at step 740, the processor can analyze the resulting imageusing an image processing algorithm to check that the resulting image isconsistent with that expected from a genuine periocular region. It canbe appreciated that the reflection of light from a genuine cornea is afunction of the curvature of the eye, which varies from the reflectionof a reproduction, say, a flat image of the cornea. As a result thepattern of light reflected (e.g., concentration) varies accordingly. Insome implementations, the image can be compared to one or more similarlygenerated images of genuine periocular regions (e.g., of the user orother users) or compared to prescribed characteristics identified fromanalyzing imagery of genuine periocular regions. For example, theprocessor can employ a haar classifier, and/or algorithm for detectingthe presence of a strong reflection peak within the region of the pupil,and of an expected size/concentration of the reflection.

Then, at step 745, the processor can calculate a confidence levelindicating the likelihood that the images are captured from a genuineperiocular region. For example, the confidence level can be a functionof how closely the resulting image matches the one or more previouslygenerated images or prescribed characteristics (e.g., as determined atstep 740). In addition, the confidence level can be a function ofwhether the intensity exhibits more constant intensity characteristic ofimaging a non-genuine periocular region or exhibits sharp peaks of highintensity corresponding to the reflection that are characteristic ofimaging a genuine periocular region (e.g., as determined at step 735).If the liveness confidence level exceeds a prescribed confidence levelthreshold, the processor can determine that the user is alive andauthenticate the user accordingly.

In other embodiments, the LED's can both be pulsated out of phase witheach other. The frequencies of the LED pulsating, and the number offrames captures may be adjusted. Pulsating light allows the system toslow down the frame rate of capture to acquire more detailed imagery.For example, pulsating the LEDs out of phase or at different frequenciescan enable the system to capture data for determining liveness invarying spectrums. Moreover pulsating LEDs at different frequencies canbe used to perform analysis in different ambient light scenarios. Forexample, outdoors where ambient IR light levels are high and indoorswhere IR levels are lower. Also bursts of IR light can be emitted andcan improve the quality of the data collected as compared to a singlestream of light and can prolong LED life. Pulsating frequency can alsobe varied so as to avoid triggering adverse physical responses fromusers, for example, epileptic reactions. Moreover, simple imagesubtraction could be used in place of pulse frequency analysis to reducethe number of frames required.

In addition or alternatively, the mobile device processor 110, executingone or more software modules 130, including, preferably, the capturemodule 172, and the analysis module 174 can capture images of the userusing the mobile device camera, and can analyze the images to detect thepresence of the user's face, for example, by using shape recognition orother known face identification techniques. Once the face is detected inthe images, the configured processor can similarly locate one or more ofthe user's eyes in the images. In addition, the configured mobile deviceprocessor 110 can cause the display (or a light emitter) to be pulsated,for example, by changing the intensity of the light emitted by thedisplay. Preferably, the intensity is pulsated over time sinusoidally ata frequency of 3 Hz, for 2 seconds. During this time the configuredprocessor, using the camera, can capture images of the eye and recordthe images, preferably, at a frame rate of at least twice the frequencyof the display pulsation (e.g., flash).

In a further embodiment, as the images are recorded, the position of theeye can be tracked so that all of the images to be analyzed will be ofthe eye and have at least a generally consistent alignment with oneother. It can be appreciated that the eye tracking can be performed inaccordance with the disclosed embodiments and/or known eye position oreye tracking algorithms, as would be understood by those in the art. Insome implementations, the configured processor can cause an animatedlogo, or information such as a news feed to be displayed during themeasurement so as to draw the user's eyes to a particular position, andentertain the user during the image capture process.

After image collection, the mobile device processor110, which isconfigured by executing one or more software modules 130, including theanalysis module 174 can perform an analysis of the images to check thatthe reflectivity characteristics of the eye are consistent with that ofa real eye and not a photograph. In some implementations, the configuredprocessor can analyze reflectivity characteristics to determine whetherthe curved surface of the cornea has produced a small, sharp, specularreflection of the display that is visible to the camera, as discussedabove. A photograph/video generally produces a diffuse reflection thatis uniform across the whole image, or a speculate reflection from a flatsurface making it much larger than from that of an eye.

In addition, each pixel of the frame sequence can be analyzed, by theconfigured processor 110, to identify the strength of the frequencycomponent at the display pulsation frequency (referred to as the “powersignal”). Thus a ‘power image’ image can be created where the intensityof each pixel is the power signal.

A power image of a real eye will contain a peak over the cornea of theeye. The presence of a peak is tested by the configured processor 110,for example, by applying a band pass filter over the image to removehigh frequency noise and the low frequency background, then finding thepeak power signal in the power image, then checking that the peak is ofthe expected size and magnitude above the background. A determination ofliveness can be made against this data.

Alternatively the power signal may be calculated as a ratio of thesignal strength at the display pulsation frequency divided by the sum ofthe signal strength at other frequencies. This means that if the signalis noisy (and other frequencies exist perhaps due to motion or motionblur), the power signal is reduced and so is discounted from the finalpower image. Such motion noise can be present during for example, eyemovement, photo movement, or movement in a video spoof.

Additionally the phase of the power signal at the display pulsationfrequency can be calculated and compared with the phase of the displaypulsation frequency. If the phase of the power signal is not in phasewith the display frequency then the configured processor 110 canconclude that the power signal must be from noise and discounted orattenuated as an indicator of liveness.

In some implementations, for speed of analysis, a Goertzel filter couldbe used, by the configured processor 110, to measure the power signal atintervals over the frequency spectrum.

In addition or alternatively the mobile device can be configured toanalyze reflectivity characteristics of imagery captured by the cameraas discussed above, except that the display can be caused to pulsate atthe sum of two (or more) frequencies, for example 3 Hz and 2 Hz, with aparticular phase difference, say 180 degrees. Accordingly, the powersignal is then calculated as the sum of the signal at 2 Hz and 3 Hzdivided by the signal at other frequencies.

The phase signal can be calculated as the difference between the phaseat 2 Hz and 3 Hz, and the more the signal's phase deviates from theexpected 180 degrees the more it is discounted or attenuated as anindicator of liveness.

In some implementations, the power image may make use of ‘super pixels’to reduce quantization noise. Quantization occurs when the intensity ofeach pixel in the image frames from the camera are stored as a discretevalue (typically an integer from 0 to 255). To reduce the negativeeffects of this quantization on the power signal, each pixel may beaveraged with surrounding pixels (e.g. using a Gaussian blur with a blurdiameter roughly equal to the width of the expected size of the displayreflection from the eye) to create a ‘super pixel’ that has lessquantization artifacts. These super pixels are stored with a greaterintensity accuracy than in the original image frames (such as with a 32bit floating point number, or 16 bit integer giving intensity valueswith 65536 steps rather than 255). This increases the quality of thepower signal that can be derived and makes the systems less prone toerror.

In a further embodiment, the mobile device can be configured to pulsatethe display and analyze reflectivity characteristics of imagery capturedby the camera as discussed above, except that the phase of the displaypulsation is different for each color channel. This has the result ofmaking the color of the display change with time, and the phase imagecan be calculated based on the expected phase difference between eachcolor channel. For example by making the red and blue channels havephase 0 degrees, and the green channel have phase 180 degrees, thedisplay will pulsate between green and magenta.

In a further embodiment, instead of or as well as the reflection peakfrom the cornea being detected the configured processor executing theanalysis algorithm checks for the presence of shadows from the displayillumination on the face and checks that they are consistent with thatof a real face and not a photograph or video. For example, this could bedone using a Haar Classifier on the power image, or a ‘shadow image’that is a combination of the power image and the power image at 0 Hz.

Although the foregoing exemplary embodiments for analyzing reflectivitycharacteristics of imagery captured by the camera to determine if thereflectivity is consistent with a live cornea, similar methods can beperformed to determine that the reflectivity is not consistent with thereflectivity of a high resolution print or video display.

One characteristic of nearly all prints and video displays is that theyare substantially flat. Therefore the specular reflection from theirsurface will be similar to that of a flat minor. When a print or videoof a face is presented (directly facing) the smartphone camera, thesmartphone camera would be able to capture a reflection of thesmartphone display in the print or video caused by the specularreflections. Such a reflection is not expected from a live personbecause human skin has a highly diffuse reflectivity and the face is notflat.

The display reflection could be detected (for example) by using methodssimilar to that used to detect display reflections from the cornea ofthe eye. For example, the display could be pulsated at a known frequencyand the display reflection could be isolated from the background byisolating intensity changes at that frequency. Similarly the displaycould show a pattern spatially and the presence of this pattern could besearched for in the camera image by the mobile device processor 110,which is configured by executing the software modules 130, including,preferably, the analysis module 174.

The reflection pattern can be compared to known reflection patterns andcharacteristics of various surfaces (e.g., a photograph or videodisplay) and if the reflection of the display is consistent with thatfrom a substantially flat surface then the configured processor candetermine the imagery is the result of a spoof attempt. Similarly if thereflection is consistent with a print or video that has been curved inonly one dimension (or other non-face like shapes) the processor canalso determine that the imagery is the result of a spoof attempt.Similarly if the reflection is consistent with that from a display witha diffusive anti-glare coating (as is used on some liquid crystaldisplay panels) the imagery is the result of a spoof attempt.

In addition or alternatively the configured mobile device processor 110can analyze the imagery to check that the reflection of the display inthe face is stronger than that from the background. To do this theconfigured processor 110 can average all the pixel intensities from theface region and look for the display flash frequency, and compare thisto the same figure for the background pixels. It can be expected for thereflection from the face to be stronger than the background because itis closer to the smartphone display and camera than the background.

At this juncture it should be noted that the foregoing exemplaryembodiments for analyzing reflectivity characteristics of imagerycaptured by the camera, so as to differentiate between a real face andphotograph or video of a face are not limited to smartphone devices andcan be applied to any device with a light source and a camera, such as alaptop computer with a webcam. In addition, the frequency of the displaypulsation (e.g. flash) could be any frequency, and the duration of themeasurement may be adjusted depending on the confidence required fromthe measurement. In addition, a lux meter could be used to measureambient light levels and increase the measurement time in high lightingsuch as sunlight. Knowledge of the ambient light levels could also beused to help determine the expected power signal. For example, in mediumlighting the configured processor might expect the strongest signalstrength, in low light levels the configured processor might expect theeye reflection peak to saturate the camera, and cause reduced signalstrength, in high ambient lighting the configured processor might expectthe signal strength to be reduced. In addition, in some implementations,the phase of the power signal in the region around the eye or the wholeface could be checked to see if it is consistent with that expected fromthe display pulsation signal. If the phase is consistent it indicatesthat ambient light is low enough for good signal to be detected from theface, if it is not consistent it indicates weak signal and thereforelikely high ambient lighting. This information can be used in place oras well as a lux meter.

In a further embodiment, liveness can be determined by detectingmovement of higher level facial features, for example, by smiledetection. Face matchers without liveness protection can be fooled byhigh resolution photographic prints. Such face images are freelyavailable for almost anyone on the internet. A smile detection systemcan recognize facial expressions, so it can request the user to ‘smileto log in’—this is something that a photograph cannot do therebyincreasing the systems security against spoof attempts.

In some implementations, the optical flow characteristics of low, mediumand high level features can be used to detect a smile or other suchfacial movement. For example, the mobile device processor 110, executingone or more software modules 130, including, preferably, the capturemodule 172, and the analysis module 174 can analyze a video sequence ofimages to determine liveness by: finding the face in the imagery, thenstabilizing the video frames of the mouth region, then splitting themouth into a left and right region, and calculating the optical flow ofeach region. In the transition to a smile the optical flow of the leftand right regions will on average move away from each other. Opticalflow analysis naturally identifies corners and edges that are associatedwith the mouth and so it provides an efficient way of analyzing themovement of the mouth. Accordingly, liveness can be detected bydetermining that the optical flow characteristics of the facial featuresthat match expected optical flow of any number of facial expressions.Alternatively a Haar cascade algorithm could be used to detect a smile.Similarly, eye brow raise detection, or blink detection could be used.

In accordance with the disclosed embodiments, gaze detection can be usedto provide a discrete liveness test by asking the user to move theireyes in a particular way and determining if the movement of the user'sgaze moves as requested. For example, the user can be asked to ‘watchthe lights switch on’, then 3 light bulbs (left middle right) each turnon at random. If the user moves their gaze to each light bulb correctlyas they illuminate then they pass the test. A photograph would fail thistest, as should a video due to the specific and randomized irismovements that are requested. This test could be used in combinationwith other liveness tests such as facial expression testing or smiledetection. Such gaze detection can be performed using the exemplarymethods for detecting the movement of low, medium and high level facialfeatures described in relation to FIG. 5 as well as using known computervision techniques described in, for example, “In the Eye of theBeholder: A Survey of Models for Eyes and Gaze,” Published in: PatternAnalysis and Machine Intelligence, IEEE Transactions on (Volume:32,Issue: 3) and Biometrics Compendium, IEEE Date of Publication: March2010 Page(s): 478-500 ISSN: 0162-8828.

In a further embodiment, liveness can be determined by detecting vitalsigns that are present in live subjects. Each time a live subject'sheart beats the subjects face pulsates. This pulsation is so small thatit cannot be detected by human eyes but it can be captured in imageryand detected using image processing of the video signal (e.g., the imagesequence captured using the video camera). Accordingly, in accordancewith the disclosed embodiments, the mobile device can be configured todetermine liveness by analyzing the imagery and determining that thecaptured subject has a pulse. In some implementations, the mobile deviceprocessor 110, executing one or more software modules 130, including,preferably, the capture module 172, and the analysis module 174 canstabilize the video images captured by the camera and use pulsedetection algorithms to determine the presence of a pulse in thesequence of images. In addition or alternatively, the configured mobiledevice processor can differentiate between photograph and live person bypulse amplitude mapping, determining the strength of the pulse signal,determining the color of the pulse signal, determining the level ofnoise in the frequency domain. Moreover, the configured processor canalso use a second mobile device camera and LED to measure a user's pulsein the finger to help identify the pulse in the face because the pulsefrom the finger is more reliable. Accordingly, using the pulse measuredfrom the finger, all other frequencies from the face can be easilyidentified as noise by the configured processor. Alternative methods fordetecting a subject's pulse from imagery are described herein and can befound in, for example, “Remote plethysmographic imaging using ambientlight”, Wim Verkruysse et al, Optics express, 22 Dec. 2008.”

The user's pulse can be assumed to be the strongest physiologicallyviable frequency component of the signal, however, a noisy signal fromthat of a photograph can also have an assumed pulse. Accordingly, forliveness detection, it is preferable to distinguish between a noisesignal from a photograph and a genuine pulse signal from a live person.In accordance with the disclosed embodiments, in some implementationsthe configured processor can distinguish between a noise signal from aphotograph and a genuine pulse signal by checking the variance of theassumed pulse frequency throughout the data. For example, if 20 secondsof video data is recorded, the assumed pulse can be calculated by theconfigured processor using data from 0 to 5 seconds, 1 to 6 seconds, 2to 7 seconds and so on. If the pulse signal is genuine the configuredprocessor should determine that there is a low variance between each ofthese measurements, if the signal is from noise a higher variance isexpected. Accordingly, this variance information can be used by theconfigured processor to distinguish between live and spoof signals.

Similarly, as a subject breathes their chest moves and this motion canalso be detected to ensure the subject is breathing. Accordingly theconfigured mobile device can be configured to detect spoof attempts byanalyzing the imagery and detecting such chest movement so as todifferentiate between a live subject and a static reproduction (e.g., aphotograph which won't exhibit such movement).

In a further embodiment, liveness can be determined by performingthree-dimensional (3D) analysis of the imagery to verify that theimagery captured is of a live subject having 3D depth as opposed to agenerally flat reproduction of the subject (e.g., using a photograph orvideo display).

In accordance with the disclosed embodiments, the mobile deviceprocessor 110, executing one or more software modules 130, including,preferably, the capture module 172, and the analysis module 174 canprompt the user to make a scanning motion around their face with thephone camera (e.g., a side to side scan, up and down scan and the like).Using the captured imagery, the configured processor can use 3D imagingtechniques to build a 3D model of the user. Whilst photos and videos areflat, a live subject is not. By requiring the authentication object tohave a 3D face like shape, it is drastically more difficult to spoof thesystem using photographs or video.

The measured optical flow of pixels and features of the one or moreimages, for example, as discussed above in relation to FIG. 5, can beused to determine liveness based on such a 3D analysis. Imagine lookingout the side of a car as one travels, the objects in the foreground movepas very quickly because they are close, whereas the distant objectsappear to move by more slowly. Similarly, a mobile device camera movingpast a face should capture imagery showing the nose move by faster thanthe eyes and the ears because it is closer. Accordingly, the opticalflow analysis of the scene can utilize this relative motion to deducehow far away the objects captured are. If the subject is a real persontheir nose is closer to the phone than their eyes, however, if thesubject is a photograph or a video then it is not. Thus, the spoof canbe detected by analyzing the optical flow to detect the distance ofvarious low, medium and/or high level facial features or a combinationof the foregoing. In addition, the configured mobile device can alsoverify that the mobile device is moving and not the face rotating byanalyzing the optical flow of the background to the face or theaccelerometers and or compass in the mobile device.

Further to the foregoing 3D analysis using imagery, in addition oralternatively, a hardware based depth sensor could be used provide depthinformation about the face. For example a structured light depth sensorcould be used such as that used in the Microsoft Kinect depth sensor byMicrosoft Inc. and the Google Tango phone by Google Inc. Similarly, atime of flight depth sensor could be used, or a stereo camera. Usingsuch devices, liveness can be deduced from just one image with a depthmap, and that it is difficult to spoof a face correctly in threedimensions.

In some implementations, the depth maps from several images could becombined to increase the accuracy and/or extent of the depth mappedregion. In addition, a depth map can be created using focus informationfrom the optical camera of the mobile device. If the focus is swept fromnear to far as video frames of the face are collected, the distance ofeach region from the camera could be deduced by identifying which of theframes is the sharpest for that region. The nearer the region's sharpestframe is to the start of the frame sequence, the nearer that region ofthe image must be to the camera.

In some implementations, an image of the face can be produced as would adepth map.

During enrolment the face would be found and the corresponding region ofthe depth map would be stored with the face image. On authentication thesimilarity of the face image can be tested and the similarity of theshape of the face would also be tested, including comparison of the sizeof the face. The comparison could involve first aligning the enrolleddepth map with the test depth map in all three dimensions usingtechniques such as Iterative Closest Point (ICP), and then assessing thequality of the match.

Other additional hardware devices could also be useful for assessingliveness. For example spoof images are likely to look different to a(near) infra-red camera than a visible spectrum camera. For example,liquid crystal displays are typically transparent in the infra-red, andmany inks have different absorption in the infra-red spectrum. Humanfaces have certain features in the infra-red spectrum that could beconfirmed by the use of infra-red imaging, for example in the infra-redirises have a higher reflectivity. By way of further example, in thedeep infra-red, the temperature profile of the face becomes apparent.Accordingly, liveness detection can be performed by the configuredprocessor by generating a temperature profile of a user and thenanalyzing the temperature profile to confirm that the face has anexpected temperature, or temperature pattern of a human face as opposedto a video display or photograph. The expected temperature can bedetermined during enrollment and/or based on known characteristics ofhuman faces and various methods for representing a human's face asdiscussed above.

Image resolution, intensity histogram analysis, eye movement detection,video scan frequency detection, and video pixel moire patterns with thecamera CCD pixels, voice recognition may also be used to distinguishbetween live and spoof images. Accordingly it can be appreciated that,in accordance with the disclosed embodiments, liveness can be detected,by the configured mobile device processor, using any combination of theforegoing exemplary liveness detection methods.

The systems and methods for authorizing access to an access-controlledenvironment are not limited in any way to the illustrated embodimentsand/or arrangements as the illustrated embodiments and/or arrangementsdescribed are merely exemplary of the systems and methods disclosedherein, which can be embodied in various forms, as appreciated by oneskilled in the art. Some alternative embodiments, arrangements andexemplary applications include the following exemplary embodiments.

In some implementations, a user mobile device 101 a configured byexecuting the one or more software modules 130 of the secureauthentication client application, for example, as a backgroundapplication, can determine that the user 124 has opened anotherapplication that enables the user to search for goods on the internetand make a purchase from an ACE, for example, iTunes by Apple Inc. Theconfigured mobile device processor 110 can also be configured todetermine, from the user profile including user defined accessrules/preferences input during enrollment, that the particularapplication is a preferred ACE. As a result, the mobile device processorcan automatically initiate the exemplary authentication processdescribed in reference to FIGS. 3-5. In addition or alternatively, ifthe user preferences can specify that the user prefers to only log -inand authenticate upon making a purchase and the mobile device processorcan initiate biometric authentication responsive to a user initiatingpurchase through the iTunes application.

To authorize the user, the mobile device processor can prompt the userto scan his/her biometrics using the mobile device camera. Then, themobile device 101 a and/or the system server 105 can determine: if theuser is biometrically authenticated, has an iTunes account and/or isauthorized to perform the transaction using the account. For example,the authentication process can include biometrically authenticating theuser using the mobile device and transmitting from the mobile device tothe system server 105 a transaction request identifying the user andincluding information concerning the ACE requiring user authorization(e.g., iTunes). If the user's biometrics are not authenticated by themobile device or the user is not authorized by the system server 105,the mobile device can alert the user with a tone. Upon successfulbiometric authentication and authorization, the system server 105 canquery the user profile created during enrollment to retrieve informationassociated with the user's transaction account (e.g., the iTunesaccount) and transmit an authorization notification confirming theuser's identity and including the user's transaction account informationnecessary to complete the one or more fields. The authorizationnotification can be transmitted to the mobile device 101 a, therebycausing the mobile device to automatically complete the required fieldsso as to complete the user login and/or complete the song purchaseaccording to the user preferences.

In another exemplary implementation, a user computing device 101 b,which has been enrolled with the secure authentication system (e.g., apersonal laptop) and is configured by executing the secureauthentication client application, can determine that the user 124 hasopened a web browser and has navigated to a social network websiterequiring user authentication. The computing device can also determinefrom a user profile stored locally that the particular website is an ACEthat is preferably accessed using the system 100. The computing devicecan also verify the site key to assure no spoofing. Responsive todetermining that the website is a preferred ACE, the configured usercomputing device 10 lb can initiate the authorization process bytransmitting an authorization request identifying the website and theuser to the system server 105. Based on the authorization request, thesystem server 105 can locate a user profile associated with theidentified user and identify an enrolled mobile device 101 a that isalso associated with the user and/or user profile. The system server canthen transmit a biometric authentication request causing the identifiedmobile device to biometrically authenticate the user. Upon biometricauthentication of the user, the mobile device can transmit a transactionrequest confirming authentication and identifying the user and mobiledevice 101 a to the system server. Using at least the transactionrequest, the system server can authorize the user to access the websiteand transmit an authorization notification to the user computing device101 b. In response to the authorization notification the user computingdevice can automatically complete the fields required to complete theuser login facilitating the user's access to their online account. Insome implementations, the authorization notification can include theuser log-in information retrieved from a user profile maintained by thesystem server 105. In addition or alternatively, the authorizationnotification can prompt the user device 101 b to retrieve the requisitelog-in information from an instance of the user's profile stored by thecomputing device 10 lb.

In another exemplary implementation, an enrolled user computing device101 b (e.g., a personal laptop), which is configured by executing thesecure authentication client application, can determine that the userhas opened a browser application and navigated to a communicationsservice provider (e.g., www.skype.com). The user computing device 101 bcan also determine whether the user profile preferences names thepayment service provider as a trusted and preferred ACE and can alsoverify the site key to assure zero spoofing. The user computing device101 b can then initiate the authorization process to authorize the userby the mobile device and the system server in authorized in accordancewith the disclosed embodiments. Upon authorization, the system server105 can transmit an authorization notification, which includes a uniqueone-time fused key to the user computing device 101 b and cause thecomputing device to automatically decrypt the fused key using acorresponding key stored by the user computing device and complete therequired fields necessary so as to complete the user login and therebyallowing the user to gain access to their online account. For example,the key can be based on the user's biometric identifier, a useridentifier, a mobile device identifier or a combination of theforegoing.

By way of further example, after log-in, the user may encounteradditional authorization points, for instance, if the user needs topurchase credits to continue using the service and selects a “PayPal”payment option. Accordingly, the user computing device 101 b executingthe secure authentication client application can again initiate userauthorization by transmitting an authorization request identifyingPayPal as the ACE to the system server 105. Upon user authorization inaccordance with the disclosed embodiments the system server can transmitan encrypted authorization notification to the user computing device 101b including the user's PayPal account information, which is stored bythe system server in the user profile, thereby causing the computingdevice to complete the transaction by automatically populating therequired fields with the received payment information and transmittingthe information to the back end server(s) associated with the ACE (e.g.paypal and/or skype).

In another exemplary implementation, the disclosed embodiments can beused to facilitate payment at a computing device 101 b, which is atransactional terminal associated with an enterprise organization, forexample, a point of sale terminal at a supermarket. In someimplementations, the user can initiate a payment transaction byselecting, on the user's mobile device 101 a, the particular supermarketfrom a list of preferred merchants stored in the user profile. Based onthe user's selection the mobile device 101 a can identify userpreferences concerning transactions conducted with the particularmerchant. For example, that the user prefers to make payments using aparticular credit card account identified in the user profile and alsoprefers to use a loyalty program account when making purchases.Accordingly, upon receipt of the user tapping a pay button on the mobiledevice user interface 115, the mobile device 101 a can prompt the userto scan his/her biometrics and the mobile device and/or system server105 can determine if the user is biometrically authenticated andauthorized to make a payment using the particular payment method. Uponsuccessful authorization, the configured mobile device using an NFCtransmitter can transmit the user's payment information and loyaltyaccount information to the NFC enabled POS device, thereby passing theuser information, payment information and loyalty member numberassociated with the user to complete the transaction.

In another implementation, the point of sale device, (e.g., computingdevice 101 b) can receive user identification information from themobile device 101 b, and can transmit an authorization request includingthe transaction information (e.g., price, taxes, etc.) and informationidentifying the user and the particular merchant to the system server105. Using the request, the system server can identify the mobile device101 a associated with the user and cause the mobile device 101 a toprompt the user to biometrically authenticate. Upon successful biometricauthentication by the mobile device 101 a, the mobile device can notifythe system server 105 by sending a transaction request which includes asecure key identifying the user and the mobile device. Using the key,the system server 105 can identify a user profile associated with theuser and the mobile device and identify a payment account for conductingthe transaction specified in the authentication request. The systemserver 105 can then query a secure data-store maintained by theenterprise organization that maintains the user's payment account toretrieve the user's payment information. The system server can processthe transaction using the payment information and upon successfulcompletion of the transaction, the system server 105 can transmit anauthorization notification to the point of sale device indicating thatthe transaction was processed. Such an implementation avoids passingsensitive financial and personal information directly to the merchantPOS device. Alternatively, the system server can transmit, to the POSdevice, an authorization notification that includes the paymentinformation thereby causing the POS device or the merchants paymentprocessing system to complete the financial transaction.

In another implementation, a computing device 101 b controlling a secureaccess point (the ACE, for example an airport security checkpoint) canbe configured to communicate with enabled mobile devices and/or thesystem server 105. In some implementations, the access point computingdevice 101 b can transmit an authorization request directly to a user'senabled mobile device 101 a. Responsive to receipt of the request, themobile device 101 a, which is configured by executing the secureauthentication client application, can biometrically authenticate theuser. Upon authentication the mobile device can transmit a transactionrequest to the system server 105 identifying the computing device 101 b,the user and the mobile device. Responsive to the transaction request,the system server 105 can authorize the user by verifying the user'sidentity and authorize passage through the access point according touser access rules gathered from the user profile or a secure data store.For example, the rules can concern travel restrictions (e.g., whetherthe traveler is not on a no-fly list) maintained in a governmentdatabase. If the system server 105 determines that the user identity isnot verified, or the access rules indicate that the user is restrictedfrom travel, an authorization notification can be transmitted to themobile device 101 a to alert the user. The system server 105 can alsotransmit an authorization notification to the computing device 101 bcontrolling access to the ACE, for example, to prevent user accessand/or to alert a guard via a display. Similarly, upon successfulauthentication, the mobile device 101 a and the access point computingdevice 101 b can be similarly notified. In addition, the system server105 can also transmit user information, for example, a name and pictureof the user 124 to the computing device 101 b for further authorizationif necessary. The authorization notification can also cause the accesspoint computing device to allow the user 124 to physically pass throughsecurity check point, for example, open a door allowing the user to walkto their gate and await boarding.

In another implementation, the computing device 101 b can be anelectronically controlled access point (e.g. a networked electroniclock) at a secure doorway configured to communicate with enabled mobiledevices 101 a and the system server 105. The access point computingdevice 101 b can transmit an authentication request directly to themobile device 101 a causing the mobile device 101 a to begin theauthorization process. Alternatively, the mobile device 101 a cantransmit a message identifying the user and the mobile device directlyto the access point computing device 101 b thereby prompting the accesspoint to transmit an authorization request to the system server 105identifying the user and the access point. Using the request, the systemserver 105 can query the user profile associated with the user toidentify the mobile device and transmit a biometric authenticationrequest causing the mobile device 101 a to begin the biometricauthentication process. Upon successful authentication, the systemserver 105 can determine from access rules associated with theparticular check-point computing device 101 b whether the user isauthorized to access the secure area and, if so, transmit anauthorization notification to the computing device 101 b causing thecheckpoint to unlock the door.

In some implementations, the user computing device (e.g., 101 b) can bea transaction terminal, for example, an ATM, configured to interact withthe system server 105. The ATM can be further configured to communicatewith the user's enabled mobile device 101 a, for example, bytransmitting information to the mobile device 101 a when the mobiledevice is within a defined range. Upon receipt of the communication fromthe ATM, the mobile device 101 a can initiate the authentication processby prompting the user 124 to authenticate. The mobile device 101 a cancapture and authenticate the user biometrics and notify the systemserver as described in relation to FIGS. 3 and 4. Accordingly, themobile device 101 a and/or the system server 105 can determine if theuser is biometrically authenticated and determine whether the user isauthorized to use the ATM (e.g., has transaction account(s) that can beaccessed using the ATM). Moreover, the mobile device and/or systemserver can query trusted databases maintained by the system server 105or an enterprise database (e.g., remote computing device 102, which is,say, operated by a bank) to perform additional security check'saccording to the user's identity to ensure the user is not restrictedfrom conducting transactions, for example, on an AML (anti-moneylaundering) or watch list. If the user is not authenticated and/or lackspermissions to perform the transaction, the user can be alerted via themobile device 101 a. In addition, the bank (e.g., remote computingdevice 102) or the ATM (e.g., computing device 101 b) can be is notifiedof error or fraud attempt. If authorized, the system server 105 cantransmit an authorization notification to the ATM 101 b and/or anassociated banking network to advance the transaction at the ATM. Forexample, advancing the transaction can include authorizing the requestedtransaction, displaying user options (e.g., withdraw cash, transferfunds, check balance, etc.), requesting further user input infurtherance of the transaction and the like as would be understood bythose skilled in the art. In this manner, the disclosed embodiments caneliminate the need for transaction cards and PIN numbers and can deterfraud. Moreover, such a system can eliminate the need for arbitrary useraccount numbers.

In another exemplary implementation, the system server 105 and/or one ormore servers and storage devices communicatively coupled thereto, can beconfigured to host an encrypted file sharing and communication platform.It can be appreciated that the encrypted file sharing platform is notlimited to storage or transmission of encrypted data files in thetraditional sense and can be applicable to transmission of anyelectronic data packet. For example, the encrypted sharing platform canbe configured to allow users to secure and transmit emails, attachmentsof any size, text chat, voice calls (VoIP), video calls, group messagesand the like.

More specifically, enrolled user devices executing the secureauthentication application can be configured to transmit encryptedmessages to other enrolled users via the system server 105. As notedabove, preferably, all communications between an enrolled user deviceand the system server can be sent via 2-way SSL secure communicationenvironment using a key that was generated during enrollment based on,for example, the user's biometric identifier, other user and/or deviceidentifiers and/or keys generated during enrollment or a combination ofthe foregoing. Using an asymmetric key made of the users own biometricdata provides a key that is unique to the user and as such is useable toassert the user's identity. Preferably, the key is further encryptedusing a 384 bit Elliptic Curve Cipher. Accordingly, the generated keys,biometric information, data and other information encrypted using thekeys are also rendered virtually unreadable, except to the systemserver.

In addition, the system server can also receive rules input by the usersusing the enrolled user computing devices (e.g., computing device 101 band/or mobile device 101 a. For example, the rules received from theuser can identity at least one other enrolled user that is approved toreceive or have access to the encrypted files or data transmissions.Accordingly, the system server can maintain records of the relationshipsbetween the users of the system and facilitate the secure sharing ofdata between authenticated users that are authorized according to theaccess rules.

For example, a user can initiate an encrypted data transfer sessionusing mobile device 101 a and designate another user as the intendedrecipient. Accordingly, the system server can cause the sending user tobe biometrically authenticated using the mobile device. If the sendinguser is biometrically authenticated, a 2 way SSL/TLS connection can beestablished between the system server and the mobile device for everysuch transaction (e.g., session or transmission) as discussed above.Once this secure connection is created, all data sent by the user acrossthe SSL/TLS layer can be encrypted using the previously mentioned keygenerated during enrollment. This provides a robust, secure method oftransport for all data types between the sending device and the systemserver.

A salient aspect of the file sharing platform is that it requiresbiometric authentication and identity assertion totransmit/store/receive or access the encrypted information, therebyproviding a high level of protection and security for the information asit passes from a user to another user via the system server. The onlydevice with an ability to decrypt the messages is the system serverwhich contains the overall algorithm used to encrypt and decryptmessages and manages the 2-Way SSL secure communications environmentwith user devices. In the event that this algorithm was made public,each user's data is still safe, because no user data needs to reside onthe system server and all information can resides with the user's ontheir devices, and only with a valid biometric authentication, under avalid 2-way SSL connection can the information be communicated betweenthe user devices and the system server.

Upon receiving the encrypted message from the sending user and, based onthe associated access rules, the system server can securely forward themessage to the intended recipient or transmit a notification to theintended recipient informing them that a secure message is waiting to bedelivered. In particular, the system server can require the intendedrecipient to be biometrically authenticated and authorized, and, ifsuccessful, the system server can decrypt the message. In addition, thesystem server can establish a 2-way SSL communication session with theintended recipient's device in order to forward the message to therecipient in a secure manner.

It can be appreciated that the encrypted file sharing platform is notlimited to sharing encrypted data files in the traditional sense and canbe applicable to transmission of any electronic message. In someimplementations, the encrypted sharing platform can be configured toallow users to secure and transmit: Email, Text Chat, Voice Calls(VoIP), Video calls, Group Messaging using any of the above, Attachmentsof any size. In addition, the platform can be configured to performother known platform functions, such as message translations, forexample, using Google Translate by Google Inc.

In some implementations, the system server 105 can include an encryptedmail server which can sit between an enterprise mail server and the restof the world, such that it is designed to decrypt and encrypt alloutgoing email from an enrolled user that is destined for otherdesignated users. In this way, integration can be very simple for anyorganization, with no need for them to modify or replace their existingmail server (except to forward all the mail to the secure Mail Server).

It can be appreciated that in some implementations, the system server105 can also maintain a history of a user's authorizations using thesystem including any and all of the information collected, and/orprocessed during the exemplary biometric authentication andauthorization processes. For example, records and details concerningfinancial transactions, purchases, etc. made by the user in accordancewith the disclosed embodiments can be stored by the system server in oneor more databases, thereby creating a financial audit trail for theuser. It should be understood that information concerning any and allaccess requests, transactions and activity can be stored by the systemserver 105.

For example, a record of a user's authorization sessions, which can eachinclude GPS and other such physical location data, can be stored by thesystem server 105, thereby creating a physical audit trail of the user.In addition, the user can be periodically prompted to authenticate withthe system server simply for the purpose of recording the user'spersonal location in an authenticated manner. The physical and financialaudit trails stored can be accessible to the user via computing devicesthat are configured to interact with system server 105. For example,using a dashboard-like interface presented by an enrolled mobile device101 a or computing device 101 b executing the secure authenticationapplication or through a web-based user interface. Using the dashboard,the user can adjust settings, preferences and specify access rules forthe audit trails (e.g., physical, financial and the like). For example,the user 124 can review and specify other individuals and organizationswho are authorized to have access to the user's audit trail data, orspecific portions of the audit trails. In addition, the user can grantconditional access to the specified organization/person according to theuser's terms, including but not limited to, use restrictions and cost.

In some implementations, the user's GPS location information can begathered by the user's mobile device 101 a or any other GPS enabledcomputing devices (e.g., computing device 101 b) that are associatedwith the user and/or an access-controlled environment accessed by theuser in accordance with the disclosed embodiments. The usage andlocation information can be stored by the system server 105 on one ormore associated datastores. For example, a GPS enabled computing device101 b can be located in the user's automobile and collect GPS locationinformation about the car's location. The location information can betransmitted to the system server 105 or directly to a database so as tomaintain a physical audit trail of GPS data for the car and computingdevice 10 lb.

By way of further example in some implementations, the system server 105can also control access to/usage of the computing device 101 b and/or anassociated ACE (e.g., the vehicle), in accordance with the disclosedembodiments. For example, by requiring biometric authentication/userauthorization before providing access to the computing device or vehicleor otherwise restricting access.

Location data can be used for a number of purposes, by example andwithout limitation, tracking the movement of fleet vehicles, monitoringusage, tracking stolen vehicles and the like. Accordingly, it can beappreciated that in some instances it is desirable to monitor and sharethe location information collected by the computing device 101 b and theassociated vehicle. However, in view of privacy concerns, users mightnot want the location to be tracked unless necessary. In view of suchprivacy concerns, in some implementations, the user 124 can specify,rules defining the extent that the location information of, say, thecomputing device 101 b, or a mobile device 101 a or other computingdevices (e.g., a dedicated automobile location tracking device) shouldbe collected or made available for monitoring by individuals/enterprisesystems. For example, the user 124 can specify that they do not wish toshare the user's location information that is collected while the useris in the vehicle, but desires the location to be monitored while theuser is not in the car (e.g., for automobile theft tracking purposes).By way of further example, if managing a fleet of cars and employees, auser 124 can specify that they wish to track the location of a vehicleincluding computing device 101 b, when an employee is in the car.

In some implementations, when the computing device 101 b is interactedwith (e.g., activated by a user, someone starts the car causing thecomputing device 101 b to begin collecting location information and thelike), the computing device can scan the user's biometrics andbiometrically authenticate the user in accordance with the disclosedembodiments. In addition or alternatively the computing device 101 b cantransmit an authorization request to the system server 105. Theauthorization request can identify the computing device 101 b and canalso include additional information, say, a gps location of thecomputing device, an identity of the user, etc. In response to therequest, the system server can determine, from the received informationand stored user profiles, that the computing device 101 b is associatedwith a user 124 and prompt an associated mobile device 101 a toauthenticate the user. By way of further example, if multiple users haveaccess to the vehicle having a tracking device (e.g., computing device101 b) the user can be required to identify themselves to the computingdevice 101 b for authorization either before or after accessing thevehicle car. Accordingly, the authentication request can identify theparticular user, such that the system server can prompt the appropriateuser's mobile device 101 a is to biometrically authenticate the user. Inaddition or alternatively, the system server 105 can notify all approvedusers such that the appropriate user can continue authentication. Inaddition or alternatively, based on the location of the computing device101 b, the system server can identify an enrolled mobile device having acorresponding location and prompt the associated user to authenticate.

In some implementations, the user can initiate the authenticationprocess using the computing device 101 b and/or the user's mobile device101 a. For example, when the user gets into a car having computingdevice 101 b the user can initiate the authentication process such thatthe user's location is not tracked by mobile device 101 a or computingdevice 101 b. In addition or alternatively, the user can be required toauthenticate before being permitted to access/activate the carassociated with the computing device 101 b (e.g., start the car).

Provided the user's identity is authenticated, the system server 105 cangrant access to the ACE (e.g., the computing device, the car and thelike) or collect/provide access to the information recorded by thosedevices in accordance with the access rules associated with the user124, the mobile device 101 a, the computing device 101 b, the ACE andthe like. For example, if the user's preferences specify that the user'slocation information can be accessed by a spouse but should not beshared with a theft monitoring company, the system server 105 can grantaccess to the spouse and deny access to the theft tracking company. Byway of further example, if an owner of a car specifies in the settingsassociated with the computing device 101 b that a particular user hasaccess to the car between 8 AM and 11 PM and that the location should becontinuously monitored while in use by the particular user, the systemserver can permit, upon successful authentication/authorization, theparticular user to access the car during the specified time window, cancontinuously monitor the location while in use, and can also provideaccess to the location information to the owner.

At this juncture, it should be noted that although much of the foregoingdescription has been directed to systems and methods for authorizing auser to access an access-controlled environment according to the user'sbiometric features, the systems and methods disclosed herein can besimilarly deployed and/or implemented in scenarios, situations, andsettings far beyond the referenced scenarios.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyimplementation or of what may be claimed, but rather as descriptions offeatures that may be specific to particular embodiments of particularimplementations. Certain features that are described in thisspecification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable subcombination. Moreover, although features may be describedabove as acting in certain combinations and even initially claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising”, when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof. It should be noted that use ofordinal terms such as “first,” “second,” “third,” etc., in the claims tomodify a claim element does not by itself connote any priority,precedence, or order of one claim element over another or the temporalorder in which acts of a method are performed, but are used merely aslabels to distinguish one claim element having a certain name fromanother element having a same name (but for use of the ordinal term) todistinguish the claim elements. Also, the phraseology and terminologyused herein is for the purpose of description and should not be regardedas limiting. The use of “including,” “comprising,” or “having,”“containing,” “involving,” and variations thereof herein, is meant toencompass the items listed thereafter and equivalents thereof as well asadditional items. It is to be understood that like numerals in thedrawings represent like elements through the several figures, and thatnot all components and/or steps described and illustrated with referenceto the figures are required for all embodiments or arrangements.

Thus, illustrative embodiments and arrangements of the present systemsand methods provide a computer implemented method, computer system, andcomputer program product for authorizing a user to access anaccess-controlled environment. The flowchart and block diagrams in thefigures illustrate the architecture, functionality, and operation ofpossible implementations of systems, methods and computer programproducts according to various embodiments and arrangements. In thisregard, each block in the flowchart or block diagrams can represent amodule, segment, or portion of code, which comprises one or moreexecutable instructions for implementing the specified logicalfunction(s). It should also be noted that, in some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

The subject matter described above is provided by way of illustrationonly and should not be construed as limiting. Various modifications andchanges can be made to the subject matter described herein withoutfollowing the example embodiments and applications illustrated anddescribed, and without departing from the true spirit and scope of thepresent invention, which is set forth in the following claims.

1-27. (canceled)
 28. An authorization system for securely coordinatingaccess to an access-controlled environment for a user using a mobilecomputing device executing a biometric authentication application tobiometrically confirm the user's identity a function of a biometric ofthe user, the system comprising: a non-transitory computer readablestorage medium having instructions in the form of one or more modulesstored therein; keys stored in the storage medium and associated withrespective user accounts, wherein each respective user account isassociated with a respective key generated based on confirmation of arespective user's identity by a respective mobile device executing thebiometric authentication application and using identificationinformation concerning the respective user and a component of the of therespective mobile device; a processor configured by executing themodules therein; a network connection configured to communicativelyconnect the processor to one or more mobile devices over a networkconnection, and wherein the processor, using the network connection, isconfigured to receive: i) access-control information that identifies anaccess-controlled environment, and ii) one or more transmissions from amobile device including a key and an indicator indicating that theuser's identity has been biometrically confirmed by the mobile device;an authorization module that, when executed by the processor, configuresthe processor to: verify that the received key corresponds to at leastone of the respective keys, determine, based on the indicator and thekey, that the mobile device has biometrically confirmed the identity ofthe user using the biometric authentication application, confirm that atleast one of the one or more transmissions is not a replay of apreviously received transmission from the mobile device, and facilitate,over the network with a remote computing device based on the informationthat identifies an access-controlled environment, the user access to theaccess-controlled environment as a function of the verification,determination and confirmation.
 29. The system of claim 28, wherein theprocessor is configured to confirm that the at least one transmission isnot a replay by verifying that the at least one transmission differsfrom one or more previous received transmissions.
 30. The system ofclaim 28, wherein a prescribed manner in which each transmissionreceived from the mobile device is unique is provided in the storage,and wherein the processor is configured to confirm that the at least onetransmission is not a replay by: verifying that the at least onetransmission conforms to the prescribed manner.
 31. The system of claim28, wherein the authentication module further configures the processorto: prompt the mobile device to: i) capture biometric information of theuser, ii) biometrically confirm the user's identity, and iii) transmitthe indicator.
 32. The system of claim 31, wherein the access-controlinformation includes information identifying the user, and wherein theauthentication module further configures the processor to: identify auser account, among the respective user accounts, that is associatedwith the user, and identity the mobile device that is associated withthe user account in the storage.
 33. The system of claim 32, furthercomprising: wherein, in response to the biometric authenticationrequest, the biometric authentication application executed by the mobiledevice, configures the mobile device to: capture biometric informationof the user; confirm the identity of the user as a function of thecaptured biometric information and biometric information previouslystored on the mobile device, and determine that the biometricinformation is representative of a live subject; and transmit the one ormore transmissions including the indicator and the key.
 34. The systemof claim 28, wherein the processor is configured to verify that the keycorresponds to at least one of the respective keys by: testing the keyagainst one or more of the respective keys to identify a match; andidentifying a user account associated with the respective key matched tothe key.
 35. The system of claim 34, wherein the one or moretransmissions include one or more additional keys, and wherein theprocessor is configured to verify that the one or more additional keyscorrespond to the user account.
 36. The system of claim 34, wherein thestorage includes an access rule associated with the user account,wherein the access rule defines the user's access to theaccess-controlled environment; and wherein the processor is configuredto: retrieve the access rule from the storage; facilitate the useraccess to the access-controlled environment based on the access rule.37. The system of claim 28, wherein the at least one remote computingdevice includes a first remote computing device that is associated withthe access-controlled environment and grants access to theaccess-controlled environment, and a second remote computing device thatis associated with the user, and wherein the processor is configured tofacilitate the user access by: establishing a secure communicationsession between the first remote computing device and the second remotecomputing device.
 38. The system of claim 37, wherein the processor isconfigured to establish the secure communication session based on thereceived key.
 39. The system of claim 37, wherein the processor isconfigured to establish the secure communication session by:transmitting one or more notifications to the first remote computingdevice and the second remote computing device, wherein the one or morenotifications causes the first remote computing device and the secondremote computing devices to establish the secure communication sessionthere-between;
 40. The system of claim 39, wherein at least one of theone or more notifications transmitted to the first remote computingdevice includes a representation of one or more of: informationidentifying the user, information identifying a user account associatedwith the user, a unique identifier for the secure communication session.41. The system of claim 40, wherein at least one of the one or morenotifications transmitted to the second remote computing device causesthe second remote computing device to: retrieve, from a memory inaccordance with the at least one notification, account detailsassociated with the access-controlled environment, and transmit at leastthe account details to the first remote computing device.
 42. The systemof claim 37, wherein the second remote computing device is the mobiledevice.
 43. A method for securely coordinating access to anaccess-controlled environment for a user that is using a mobilecomputing device executing a biometric authentication application tobiometrically confirm the user's identity a function of a biometric ofthe user, the method comprising: receiving, at a computing device over anetwork connection, one or more transmissions from a mobile deviceincluding: i) access-control information that identifies anaccess-controlled environment, and ii) one or more transmissions from amobile device including a key and an indicator indicating that theuser's identity has been biometrically confirmed by the mobile device,wherein the computing device has access to a non-transitory computerreadable storage medium having keys stored therein including respectivekeys stored in association with respective user accounts, wherein eachrespective key is generated based on confirmation of a respective user'sidentity by a respective mobile device and using identificationinformation concerning the respective user and a component of the of therespective mobile device; verifying, using the code executing in theprocessor, that the received key corresponds to at least one of therespective keys; determining, using the code executing in the processorbased on the indicator and the key, that the mobile device hasbiometrically confirmed the identity of the user using the biometricauthentication application; confirming, using the code executing in theprocessor, that at least one of the one or more transmissions is not areplay of a previously received transmission from the mobile device;based on the verifying, determining and confirming steps and thereceived access-control information, facilitating the user access to theaccess-controlled environment, using the code executing in the processorin conjunction with one or more remote computing devices.
 44. The methodof claim 43, wherein the step of facilitating the user access comprises:establishing, using the code executing in the processor, a securecommunication session between a first remote computing device associatedwith the access controlled environment and a second remote computingdevices associated with the user.
 45. The method of claim 44, furthercomprising: identifying, using the code executing in the processor basedon the information that identifies an access-controlled environment, thefirst remote computing device; and identifying, using the code executingin the processor based on the information that identifies anaccess-controlled environment and the received key, the second remotecomputing device associated with the user.
 46. The method of claim 44,wherein the step of establishing the secure communication comprises:transmitting one or more notifications to the first remote computingdevice and the second remote computing device, wherein the one or morenotifications causes the first remote computing device and the secondremote computing devices to establish the secure communication sessionthere-between.
 47. The method of claim 46, wherein the one or morenotifications transmitted includes one or more of: informationidentifying the user, information identifying a user account associatedwith the user, a unique identifier for the secure communication session.